Data Flow Flex Runner Fails on start

I created a simple beam pipeline that looks like this

with beam.Pipeline(options=options) as pipeline:      (pipeline | 'readFile' >> ReadFromText(..)               | 'parseJson' >> beam.Map(json.loads)               | 'convertToRow' >> beam.Map(lambda x: beam.Row(a=str(x['a'])))               | 'sql' >> SqlTransform(""" SELECT a FROM PCOLLECTION """)               | 'print' >> beam.Map(print)     )    

and followed the code examples provided here: https://cloud.google.com/dataflow/docs/guides/templates/using-flex-templates to be able to run this as a flex template. This is the error that I see

FileNotFoundError: [Error 2] No such file or directory: 'java': 'java'  

My (very limited) understanding is that the docker image created as part of the flex template just launches the job on Dataflow, so don't quite understand why it's complaining about java directory not being present. Any leads would be greatly appreciated.

https://stackoverflow.com/questions/66682878/data-flow-flex-runner-fails-on-start March 18, 2021 at 07:48AM