Pandas convert column values to columns

I have a dataframe that is 24 columns wide and 185,912 rows and would like to transform the dataframe such that the values in one column (name) become new columns with the values from another column (s_counter).

    uuid    start_time  s_counter   name  0   5f7c88804c4b170001683c13    2020-10-06 8:35:15  16.0  statistics$𝑒𝑥𝑒𝑐𝑢𝑡𝑖𝑜𝑛𝑠$total  1   5f7c88804c4b170001683c13    2020-10-06 8:35:15  12.0  statistics$𝑒𝑥𝑒𝑐𝑢𝑡𝑖𝑜𝑛𝑠$passed  2   5f7c88804c4b170001683c13    2020-10-06 8:35:15  4.0   statistics$𝑒𝑥𝑒𝑐𝑢𝑡𝑖𝑜𝑛𝑠$skipped  3   5f7c88804c4b170001683c13    2020-10-06 8:35:15  4.0   statistics$𝑑𝑒𝑓𝑒𝑐𝑡𝑠$to_investigate$total  4   5f7c88804c4b170001683c13    2020-10-06 8:35:15  4.0   statistics$𝑑𝑒𝑓𝑒𝑐𝑡𝑠$to_investigate$ti001  

Should become

  uuid start_time statistics$𝑒𝑥𝑒𝑐𝑢𝑡𝑖𝑜𝑛𝑠$total statistics$𝑒𝑥𝑒𝑐𝑢𝑡𝑖𝑜𝑛𝑠$passed statistics$𝑒𝑥𝑒𝑐𝑢𝑡𝑖𝑜𝑛𝑠$skipped statistics$𝑑𝑒𝑓𝑒𝑐𝑡𝑠$to_investigate$ti001  0 5f7c88804c4b170001683c13 16.0 12.0 4.0 4.0 4.0  

Also acceptable would be if we retained the 4 rows and the newly created columns contained NULL where there was no value.

Note in the larger dataframe some of these columns may not be present, i.e. if there are no values for statistics$𝑒𝑥𝑒𝑐𝑢𝑡𝑖𝑜𝑛𝑠$skipped then this entry would not be available for that particiular uuid, in these cases we would still need/want the column it would simply have a NULL or 0 value.

https://stackoverflow.com/questions/66036555/pandas-convert-column-values-to-columns February 04, 2021 at 06:19AM