Revisar Crawler para tratar erros de conexão com o Opensearch
Nos casos em que o Opensearch não responde (erro de rede ou no momento do backup), a gravação dos tweets é interrompida.
Erro ConnectionTimeout caused by - ReadTimeoutError(HTTPSConnectionPool(host=xxxx, port=443): Read timed out. (read timeout=60)) Traceback (most recent call last): File "/var/webapp/twitsearch/lib/python3.8/site-packages/urllib3/connectionpool.py", line 468, in _make_request six.raise_from(e, None) File "", line 3, in raise_from File "/var/webapp/twitsearch/lib/python3.8/site-packages/urllib3/connectionpool.py", line 463, in _make_request httplib_response = conn.getresponse() File "/usr/lib/python3.8/http/client.py", line 1348, in getresponse response.begin() File "/usr/lib/python3.8/http/client.py", line 316, in begin version, status, reason = self._read_status() File "/usr/lib/python3.8/http/client.py", line 277, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
- Não realizar a captura entre 0h e 1h da manhã
- Tratar o erro para interromper a carga de forma "elegante", interrompendo os processos seguintes.