fixed request_id error, but got new error
usage: create_configs.py [-h] [--bucket BUCKET] [--cluster_name CLUSTER_NAME]
[--dry_run DRY_RUN]
[--dataproc_version DATAPROC_VERSION]
[--spark_version SPARK_VERSION]
[--hadoop_version HADOOP_VERSION] [--region REGION]
[--user_name USER_NAME] [--os_user OS_USER]
[--pip_mirror PIP_MIRROR] [--application APPLICATION]
create_configs.py: error: argument --pip_mirror: expected one argument
sudo -S -p '[sudo] password: ' \cp /tmp/create_configs.py /usr/local/bin/create_configs.py
sudo -S -p '[sudo] password: ' chmod 755 /usr/local/bin/create_configs.py
sudo -S -p '[sudo] password: ' mkdir -p /usr/lib/python3.8/datalab/
mkdir -p /home/datalab-user/datalab_libs/
scp -i /root/keys/DLab-Dev-key.pem /usr/lib/python3.8/datalab/*.py datalab-user@172.31.16.2:/home/datalab-user/datalab_libs/
chmod a+x /home/datalab-user/datalab_libs/*
sudo -S -p '[sudo] password: ' mv /home/datalab-user/datalab_libs/* /usr/lib/python3.8/datalab/
sudo -S -p '[sudo] password: ' rm -rf /home/datalab-user/datalab_libs/
sudo -S -p '[sudo] password: ' mkdir -p /usr/lib64/python3.8
sudo -S -p '[sudo] password: ' ln -fs /usr/lib/python3.8/datalab /usr/lib64/python3.8/datalab
sudo -S -p '[sudo] password: ' bash -l -c 'echo "[global]" > /etc/pip.conf; echo "proxy = $(cat /etc/profile | grep proxy | head -n1 | cut -f2 -d=)" >> /etc/pip.conf'
sudo -S -p '[sudo] password: ' bash -l -c 'echo "use_proxy=yes" > ~/.wgetrc; proxy=$(cat /etc/profile | grep proxy | head -n1 | cut -f2 -d=); echo "http_proxy=$proxy" >> ~/.wgetrc; echo "https_proxy=$proxy" >> ~/.wgetrc'
sudo S -p '[sudo] password: ' bash -l -c 'unset http_proxy https_proxy; export gcp_project_id="or2-msq-epmc-dlab-t1iylu"; export conf_resource="dataengine-service"; /usr/bin/python3 /usr/local/bin/create_configs.py --bucket lfr-gc-23-11-proj1-local-bucket --cluster_name lfr-gc-23-11-proj1-local-des-dp1 --dataproc_version 2.0.0-RC22-ubuntu18 --spark_version 3.0.1 --hadoop_version 3.2.1 --region us-west1 --user_name proj1 --os_user datalab-user --pip_mirror -application rstudio'
Traceback (most recent call last):
File "/root/scripts/rstudio_install_dataengine-service_kernels.py", line 75, in <module>
conn.sudo('''bash -l -c 'unset http_proxy https_proxy; export gcp_project_id="{0}"; export conf_resource="{1}"; /usr/bin/python3 /usr/local/bin/create_configs.py --bucket {2} --cluster_name {3} --dataproc_version {4} --spark_version {5} --hadoop_version {6} --region {7} --user_name {8} --os_user {9} --pip_mirror {10} --application {11}' '''
File "<decorator-gen-4>", line 2, in sudo
File "/usr/local/lib/python3.8/dist-packages/fabric/connection.py", line 30, in opens
return method(self, *args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/fabric/connection.py", line 737, in sudo
return self._sudo(self._remote_runner(), command, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/invoke/context.py", line 213, in _sudo
return runner.run(cmd_str, watchers=watchers, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/invoke/runners.py", line 380, in run
return self._run_body(command, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/invoke/runners.py", line 442, in _run_body
return self.make_promise() if self._asynchronous else self._finish()
File "/usr/local/lib/python3.8/dist-packages/invoke/runners.py", line 509, in _finish
raise UnexpectedExit(result)
invoke.exceptions.UnexpectedExit: Encountered a bad command exit code!
Command: 'sudo S -p \'[sudo] password: \' bash -l -c \'unset http_proxy https_proxy; export gcp_project_id="or2-msq-epmc-dlab-t1iylu"; export conf_resource="dataengine-service"; /usr/bin/python3 /usr/local/bin/create_configs.py --bucket lfr-gc-23-11-proj1-local-bucket --cluster_name lfr-gc-23-11-proj1-local-des-dp1 --dataproc_version 2.0.0-RC22-ubuntu18 --spark_version 3.0.1 --hadoop_version 3.2.1 --region us-west1 --user_name proj1 --os_user datalab-user --pip_mirror -application rstudio\' '
Exit code: 2
fixed request_id error, but got new error
usage: create_configs.py [-h] [--bucket BUCKET] [--cluster_name CLUSTER_NAME]
[--dry_run DRY_RUN]
[--dataproc_version DATAPROC_VERSION]
[--spark_version SPARK_VERSION]
[--hadoop_version HADOOP_VERSION] [--region REGION]
[--user_name USER_NAME] [--os_user OS_USER]
[--pip_mirror PIP_MIRROR] [--application APPLICATION]
create_configs.py: error: argument --pip_mirror: expected one argument
sudo -S -p '[sudo] password: ' \cp /tmp/create_configs.py /usr/local/bin/create_configs.py
sudo -S -p '[sudo] password: ' chmod 755 /usr/local/bin/create_configs.py
sudo -S -p '[sudo] password: ' mkdir -p /usr/lib/python3.8/datalab/
mkdir -p /home/datalab-user/datalab_libs/
scp -i /root/keys/DLab-Dev-key.pem /usr/lib/python3.8/datalab/*.py datalab-user@172.31.16.2:/home/datalab-user/datalab_libs/
chmod a+x /home/datalab-user/datalab_libs/*
sudo -S -p '[sudo] password: ' mv /home/datalab-user/datalab_libs/* /usr/lib/python3.8/datalab/
sudo -S -p '[sudo] password: ' rm -rf /home/datalab-user/datalab_libs/
sudo -S -p '[sudo] password: ' mkdir -p /usr/lib64/python3.8
sudo -S -p '[sudo] password: ' ln -fs /usr/lib/python3.8/datalab /usr/lib64/python3.8/datalab
sudo -S -p '[sudo] password: ' bash -l -c 'echo "[global]" > /etc/pip.conf; echo "proxy = $(cat /etc/profile | grep proxy | head -n1 | cut -f2 -d=)" >> /etc/pip.conf'
sudo -S -p '[sudo] password: ' bash -l -c 'echo "use_proxy=yes" > ~/.wgetrc; proxy=$(cat /etc/profile | grep proxy | head -n1 | cut -f2 -d=); echo "http_proxy=$proxy" >> ~/.wgetrc; echo "https_proxy=$proxy" >> ~/.wgetrc'
sudo
S -p '[sudo] password: ' bash -l -c 'unset http_proxy https_proxy; export gcp_project_id="or2-msq-epmc-dlab-t1iylu"; export conf_resource="dataengine-service"; /usr/bin/python3 /usr/local/bin/create_configs.py --bucket lfr-gc-23-11-proj1-local-bucket --cluster_name lfr-gc-23-11-proj1-local-des-dp1 --dataproc_version 2.0.0-RC22-ubuntu18 --spark_version 3.0.1 --hadoop_version 3.2.1 --region us-west1 --user_name proj1 --os_user datalab-user --pip_mirror-application rstudio'Traceback (most recent call last):
File "/root/scripts/rstudio_install_dataengine-service_kernels.py", line 75, in <module>
conn.sudo('''bash -l -c 'unset http_proxy https_proxy; export gcp_project_id="{0}"; export conf_resource="{1}"; /usr/bin/python3 /usr/local/bin/create_configs.py --bucket {2} --cluster_name {3} --dataproc_version {4} --spark_version {5} --hadoop_version {6} --region {7} --user_name {8} --os_user {9} --pip_mirror {10} --application {11}' '''
File "<decorator-gen-4>", line 2, in sudo
File "/usr/local/lib/python3.8/dist-packages/fabric/connection.py", line 30, in opens
return method(self, *args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/fabric/connection.py", line 737, in sudo
return self._sudo(self._remote_runner(), command, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/invoke/context.py", line 213, in _sudo
return runner.run(cmd_str, watchers=watchers, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/invoke/runners.py", line 380, in run
return self._run_body(command, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/invoke/runners.py", line 442, in _run_body
return self.make_promise() if self._asynchronous else self._finish()
File "/usr/local/lib/python3.8/dist-packages/invoke/runners.py", line 509, in _finish
raise UnexpectedExit(result)
invoke.exceptions.UnexpectedExit: Encountered a bad command exit code!
Command: 'sudo
S -p \'[sudo] password: \' bash -l -c \'unset http_proxy https_proxy; export gcp_project_id="or2-msq-epmc-dlab-t1iylu"; export conf_resource="dataengine-service"; /usr/bin/python3 /usr/local/bin/create_configs.py --bucket lfr-gc-23-11-proj1-local-bucket --cluster_name lfr-gc-23-11-proj1-local-des-dp1 --dataproc_version 2.0.0-RC22-ubuntu18 --spark_version 3.0.1 --hadoop_version 3.2.1 --region us-west1 --user_name proj1 --os_user datalab-user --pip_mirror-application rstudio\' 'Exit code: 2