AWS Glue | API call | Python Shell | Connection | Failed to establish a new connection: [Errno 110] Connection timed out

We are trying to call An API using Python Shell in AWS glue. We are making an HTTPS call using python 3.6 using "requests" library. The issue is pertaining to AWS glue.

The Job config is Python Shell Python 3 ( Glue Version 1.0 )

This is the error that we get

Traceback (most recent call last):    File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 160, in _new_conn      (self._dns_host, self.port), self.timeout, **extra_kw    File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 84, in create_connection      raise err    File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 74, in create_connection      sock.connect(sa)  TimeoutError: [Errno 110] Connection timed out    During handling of the above exception, another exception occurred:    Traceback (most recent call last):    File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen      chunked=chunked,    File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 381, in _make_request      self._validate_conn(conn)    File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 978, in _validate_conn      conn.connect()    File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 309, in connect      conn = self._new_conn()    File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 172, in _new_conn      self, "Failed to establish a new connection: %s" % e  urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f862b3b4160>: Failed to establish a new connection: [Errno 110] Connection timed out    During handling of the above exception, another exception occurred:    Traceback (most recent call last):    File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 449, in send      timeout=timeout    File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 727, in urlopen      method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]    File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 446, in increment      raise MaxRetryError(_pool, url, error or ResponseError(cause))  urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='xyz.com', port=443): Max retries exceeded with url: /api-v2/client/orders/?auth_token=#####&start_date=01-04-2021&end_date=02-04-2021&status_filter=all&page_number=1 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f862b3b4160>: Failed to establish a new connection: [Errno 110] Connection timed out',))    During handling of the above exception, another exception occurred:    Traceback (most recent call last):    File "/tmp/runscript.py", line 123, in <module>      runpy.run_path(temp_file_path, run_name='__main__')    File "/usr/local/lib/python3.6/runpy.py", line 263, in run_path      pkg_name=pkg_name, script_name=fname)    File "/usr/local/lib/python3.6/runpy.py", line 96, in _run_module_code      mod_name, mod_spec, pkg_name, script_name)    File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code      exec(code, run_globals)    File "/tmp/glue-python-scripts-fhh0x612/test.py", line 16, in <module>    File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 75, in get      return request('get', url, params=params, **kwargs)    File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 60, in request      return session.request(method=method, url=url, **kwargs)    File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 533, in request      resp = self.send(prep, **send_kwargs)    File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 646, in send      r = adapter.send(request, **kwargs)    File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 516, in send      raise ConnectionError(e, request=request)  requests.exceptions.ConnectionError: HTTPSConnectionPool(host='xyz.com', port=443): Max retries exceeded with url: /api-v2/client/orders/?auth_token=#####0&start_date=01-04-2021&end_date=02-04-2021&status_filter=all&page_number=1 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f862b3b4160>: Failed to establish a new connection: [Errno 110] Connection timed out',))    During handling of the above exception, another exception occurred:    Traceback (most recent call last):    File "/tmp/runscript.py", line 142, in <module>      raise e_type(e_value).with_traceback(new_stack)    File "/tmp/glue-python-scripts-fhh0x612/test.py", line 16, in <module>    File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 75, in get      return request('get', url, params=params, **kwargs)    File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 60, in request      return session.request(method=method, url=url, **kwargs)    File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 533, in request      resp = self.send(prep, **send_kwargs)    File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 646, in send      r = adapter.send(request, **kwargs)    File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 516, in send      raise ConnectionError(e, request=request)  requests.exceptions.ConnectionError: HTTPSConnectionPool(host='xyz.com', port=443): Max retries exceeded with url: /api-v2/client/orders/?auth_token=####&start_date=01-04-2021&end_date=02-04-2021&status_filter=all&page_number=1 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f862b3b4160>: Failed to establish a new connection: [Errno 110] Connection timed out',))  

Is there an issue with the security group that AWS Glue refers to or some issue with the API call. We are not able to figure out.

https://stackoverflow.com/questions/67276196/aws-glue-api-call-python-shell-connection-failed-to-establish-a-new-conn April 27, 2021 at 10:28AM