Convert unix epoch time to date/time with timezone

I have many csv files containing Unix epoch time which needs to be converted to human readable date/time. The following Python code does the job but it is very slow.

df['dt'] = pd.to_datetime(df['epoch'], unit='s')  df['dt'] = df.apply(lambda x: x['dt'].tz_localize('UTC').tz_convert('Europe/Amsterdam'), axis=1)  

Actually, the second line is the bottleneck (~30 seconds for 1 million rows). So even with the aid of multiprocessing, it is not scalable as I have more than a billion records totally. How can I make it faster?

https://stackoverflow.com/questions/65948018/convert-unix-epoch-time-to-date-time-with-timezone January 29, 2021 at 10:36AM