Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-10923

Python requirements installation in docker container is flaky

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Triage Needed
    • Priority: P1
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: test-failures

      Description

      39133  ---> 4d9c74917b64                                                              
       39134 Step 7/22 : RUN     pip install -r /tmp/base_image_requirements.txt &&     python -c "from google.protobuf.internal import api_implementation; assert api_implementation.                      _default_implementation_type == 'cpp'; print ('Verified fast protobuf used.')" &&     rm -rf /root/.cache/pip
       39135  ---> Running in 0422f8365726                                                   
       39136 Ignoring avro: markers 'python_version <= "2.7"' don't match your environment   
       39137 Ignoring futures: markers 'python_version < "3.0"' don't match your environment 
       39138 Ignoring pyvcf: markers 'python_version < "3.0"' don't match your environment   
       39139 Ignoring typing: markers 'python_full_version < "3.5.3"' don't match your environment
       39140 Ignoring guppy: markers 'python_version <= "2.7"' don't match your environment  
       39141 Ignoring numpy: markers 'python_version <= "2.7"' don't match your environment  
       39142 Ignoring scipy: markers 'python_version <= "2.7"' don't match your environment  
       39143 Ignoring pandas: markers 'python_version <= "2.7"' don't match your environment 
       39144 Ignoring tensorflow: markers 'python_version <= "2.7"' don't match your environment
       39145 Collecting avro-python3==1.8.2                                                  
       39146   Downloading avro-python3-1.8.2.tar.gz (36 kB)                                 
       39147 Collecting fastavro==0.23.3                                                     
       39148   Downloading fastavro-0.23.3-cp37-cp37m-manylinux2010_x86_64.whl (1.4 MB)      
       39149 Collecting crcmod==1.7                                                          
       39150   Downloading crcmod-1.7.tar.gz (89 kB)                                         
       39151 Collecting dill==0.3.1.1                                                        
       39152   Downloading dill-0.3.1.1.tar.gz (151 kB)                                      
       39153 Collecting future==0.18.2                                                       
       39154   Downloading future-0.18.2.tar.gz (829 kB)                                     
       39155 Collecting grpcio==1.29.0                                                       
       39156   Downloading grpcio-1.29.0-cp37-cp37m-manylinux2010_x86_64.whl (3.0 MB)        
       39157 Collecting hdfs==2.5.8                                                          
       39158   Downloading hdfs-2.5.8.tar.gz (41 kB)                                         
       39159 Collecting httplib2==0.12.0                                                     
       39160   Downloading httplib2-0.12.0.tar.gz (218 kB)                                   
       39161 Collecting mock==2.0.0                                                          
       39162   Downloading mock-2.0.0-py2.py3-none-any.whl (56 kB)                           
       39163 Collecting oauth2client==3.0.0                                                  
       39164   Downloading oauth2client-3.0.0.tar.gz (77 kB)                                 
       39165 Collecting protobuf==3.12.2                                                     
       39166   Downloading protobuf-3.12.2-cp37-cp37m-manylinux1_x86_64.whl (1.3 MB)         
       39167 Collecting pyarrow==0.16.0                                                      
       39168   Downloading pyarrow-0.16.0-cp37-cp37m-manylinux2014_x86_64.whl (63.1 MB)      
       39169 ^[[91mERROR: Exception:                                                         
       39170 Traceback (most recent call last):                                              
       39171   File "/usr/local/lib/python3.7/site-packages/pip/_vendor/urllib3/response.py", line 437, in _error_catcher
       39172     yield                                                                       
       39173   File "/usr/local/lib/python3.7/site-packages/pip/_vendor/urllib3/response.py", line 519, in read
       39174     data = self._fp.read(amt) if not fp_closed else b""                         
       39175   File "/usr/local/lib/python3.7/site-packages/pip/_vendor/cachecontrol/filewrapper.py", line 62, in read
       39176     data = self.__fp.read(amt)                                                  
       39177   File "/usr/local/lib/python3.7/http/client.py", line 461, in read             
       39178     n = self.readinto(b)                                                        
       39179   File "/usr/local/lib/python3.7/http/client.py", line 505, in readinto         
       39180     n = self.fp.readinto(b)                                                     
       39181   File "/usr/local/lib/python3.7/socket.py", line 589, in readinto              
       39182     return self._sock.recv_into(b)                                              
       39183   File "/usr/local/lib/python3.7/ssl.py", line 1071, in recv_into               
       39184     return self.read(nbytes, buffer)                                            
       39185   File "/usr/local/lib/python3.7/ssl.py", line 929, in read                     
       39186     return self._sslobj.read(len, buffer)                                       
       39187 ConnectionResetError: [Errno 104] Connection reset by peer                      
       39188                                                                      
      

      We should try to make it more resilient either during the build or restart the container build if it was unsuccessful for a few times.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                tvalentyn Valentyn Tymofieiev
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: