Google Cloud App Engine - gsutil access issue

I am looking for help to troubleshoot my application written in Python3/Flask to be hosted on Google App Engine. The deployment succeeds, but there are issues with App Engine not able to access GCS.

The code is as below:

gs_util_command = "gsutil "  gs_util_command =  gs_util_command + " cp gs://"+ re.match(r"https.//storage.googleapis.com/(.*)",url1).group(1)  gs_util_command = gs_util_command + " "+str(destination_file_path)  

I keep receiving the error message as below

Error messages: ServiceException: 401 Anonymous caller does not have storage.objects.get access to the Google Cloud Storage object.

The IAM confirms that the service has owner access to the GCS storage bucket, but it still does not work.

I tried running the below statement inside the Python3 code and deploy this to GCP:

  os.system("gsutil version -l")  

I get the below message in the shell log after deployment and execution on cloud:

2021-04-12 01:36:18 default[20210412t013030] gsutil version: 4.61 2021-04-12 01:36:18 default[20210412t013030] checksum: PACKAGED_GSUTIL_INSTALLS_DO_NOT_HAVE_CHECKSUMS (!= 61834ed701604b43c83de6cfae9b36a4) 2021-04-12 01:36:18 default[20210412t013030] boto version: 2.49.0 2021-04-12 01:36:18 default[20210412t013030] python version: 3.7.10 (default, Mar 23 2021, 05:09:08) [GCC 7.5.0] 2021-04-12 01:36:18 default[20210412t013030] OS: Linux 4.4.0 2021-04-12 01:36:18 default[20210412t013030] multiprocessing available: True 2021-04-12 01:36:18 default[20210412t013030] using cloud sdk: False 2021-04-12 01:36:18 default[20210412t013030] pass cloud sdk credentials to gsutil: False 2021-04-12 01:36:18 default[20210412t013030] config path(s): No config found

When i am trying to run the same command locally through Google Cloud Shell, it works fine.

gsutil version -l  

The output is:

gsutil version: 4.60 checksum: 4e34e1f62b6d54d4826625581f787044 (OK) boto version: 2.49.0 python version: 3.7.3 (default, Jan 22 2021,  

20:04:44) [GCC 8.3.0] OS: Linux 5.4.89+ multiprocessing available: True using cloud sdk: True pass cloud sdk credentials to gsutil: True config path(s): /home/jobsage_sai/.boto gsutil path: /usr/lib/google-cloud-sdk/bin/gsutil compiled crcmod: False installed via package manager: False editable install: False

All of this is within the virtual environment.

Please suggest.

Arvind R.

https://stackoverflow.com/questions/67051903/google-cloud-app-engine-gsutil-access-issue April 12, 2021 at 10:18AM