I have a spark dataset with column name 'T_code' which has "first.second.third xyz", I should do the below operations
T_code_op1 = if (T_code != 'N/A' and string_index(T_code, '.' > 1) then string_split(T_code, "")
T_Code_1 = if (T_code != 'N/A') then string_split(T_code_op1[0], '')
T_Code_2 = if (T_code != 'N/A' ) then T_Code_1[0] else "Temp"
T_code 1 row value: "first.second.third xyz"
Can someone tell the best to do the operations either using spark SQL or directly performing operations on the dataset.
from Recent Questions - Stack Overflow https://stackoverflow.com/questions/65376916/spark-dataset-java-deriving-new-columns-based-on-some-conditions Bhargav Lenka http://ifttt.com/images/no_image_card.png
没有评论:
发表评论