[BEAM-2587] Build fails due to python sdk

• Type: Bug
• Status: Resolved
• Priority: P2
• Resolution: Won't Fix
• Affects Version/s: 2.1.0
• Fix Version/s: 2.1.0
• Component/s: sdk-py-core
• Assignee: Unassigned
• Reporter: Ahmet Altay
• Created: 11/Jul/17 19:10
• Updated: 24/Jul/20 20:11
• Resolved: 10/Apr/18 18:08

Description

Build fails with the following errors when mvn clean package is used on a clean Ubuntu 16.04 LTS machine with pip 8.x. The issue is resolved when pip is upgraded to pip 9.x

"RuntimeError: Not in apache git tree; unable to find proto definitions."
"DistutilsOptionError: can't combine user with prefix, exec_prefix/home, or install_(plat)base​"

We need to understand the issue and maybe add a note about requiring pip 9.x for development. Note that this does not affect end users using prepackaged artifacts from central repositories.

cc: Robert Bradshaw

Script for reproduction:

#!/bin/bash

set -e

readonly MACHINE_ID=$(hexdump -n 1 -e '"%x"' /dev/random)
readonly MACHINE="${USER}-beam-build-${MACHINE_ID}"
readonly ZONE="us-central1-c"

# provision building machine
echo "Provisioning Build Machine (Ubuntu 16.04 LTS)"
gcloud compute instances create "$MACHINE" \
  --zone="$ZONE" \
  --image-project="ubuntu-os-cloud" \
  --image-family="ubuntu-1604-lts"

# wait for ssh to be ready
echo "Waiting for machine to finish booting"
sleep 30

# ssh into the machine
# 1. install dependencies as specified by beam readme
# 2. download beam source from github
# 3. build with maven
echo "Downloading and building Apache Beam (release-2.1.0)"
gcloud compute ssh "$MACHINE" --zone="$ZONE" << EOF
sudo apt-get --assume-yes update
sudo apt-get --assume-yes install \
    openjdk-8-jdk \
    maven \
    python-setuptools \
    python-pip
wget https://github.com/apache/beam/archive/release-2.1.0.tar.gz
tar -xzf release-2.1.0.tar.gz
cd beam-release-2.1.0
mvn clean package
EOF