2020年12月19日星期六

Spark Dataset Java deriving new columns based on some conditions

I have a spark dataset with column name 'T_code' which has "first.second.third xyz", I should do the below operations

T_code_op1 = if (T_code != 'N/A' and string_index(T_code, '.' > 1) then string_split(T_code, "")

T_Code_1 = if (T_code != 'N/A') then string_split(T_code_op1[0], '')

T_Code_2 = if (T_code != 'N/A' ) then T_Code_1[0] else "Temp"

T_code 1 row value: "first.second.third xyz"

Can someone tell the best to do the operations either using spark SQL or directly performing operations on the dataset.



from Recent Questions - Stack Overflow https://stackoverflow.com/questions/65376916/spark-dataset-java-deriving-new-columns-based-on-some-conditions Bhargav Lenka http://ifttt.com/images/no_image_card.png

没有评论:

发表评论