Details
Major Description
Scenario
I'm importing a PDI project.
Project has n number of job/transformation executor step referencing another job or transformation.
most if not all references use the following variables to det paths:
${Internal.Entry.Current.Folder}
${Internal.Entry.Current.File}
${Internal.Entry.Current.Directory}
Expected result
Expect that import to use either
- same variables
- or replacement variables
- replacement variables + relative path
that allow the transformation to be called
**
Actual result
It seems the import process is prepending PROJECT_HOME to the variables above which results in broken paths that prevent the pipelie/workflow from being executed.
${PROJECT_HOME}/${Internal.Entry.Current.Folder}/test_runner_db.hpl
results in
/Users/USERNAME/Documents/hop_conf/projects/PROJECTNAME/file:///Users*/USERNAME*/Documents/hop_conf/projects/*PROJECTNAME*/ETL/b3_test_runner/test_runner_db.hpl
General information
I tried this using Hop nightly build hop-client-0.70-20210325.183934-1.zip on Mac Catalina
Also below conversation from beginners channel in Mattermost
dmainou
7:39 PM
Hi team,
I imported a project and I'm running into a minor thing. Just wandering
a) if this is expected
b) what is best practice to fix it.
The path to an imported pipeline is showing as follows?
${PROJECT_HOME}/${Internal.Entry.Current.Folder}/test_runner_db.hpl
the above results in
/Users/USERNAME/Documents/hop_conf/projects/PROJECTNAME/file:///Users/USERNAME/Documents/hop_conf/projects/PROJECTNAME/ETL/b3_test_runner/test_runner_db.hpl
Image Pasted at 2021-4-7 19-39.png
New Messages
bartmaer
7:43 PM
what was the reference to the ktr file in the original job?
the "Internal" vars will lose their purpose in a lot of cases, but just removing them in the importer seems like a bit too brute force as an approach
matt
7:46 PM
${Internal.Entry.Current.Folder} should work as before @bartmaer
bartmaer
7:47 PM
yes, I agree, but there may be some unexpected results if it is now used in combination with a ${PROJECT_HOME}
matt
7:47 PM
That project home variable surely is something we are injecting?
bartmaer
7:48 PM
yes
matt
7:49 PM
Ah, we're not actually calculating the path to the .ktr.
Maybe avoid the injection in the particular case of finding ${Internal.Entry.Current.Folder}
bartmaer
7:53 PM
that's an option, but in a lot of cases, the internal vars more or less are a deprecated version of ${PROJECT_HOME}
matt
7:53 PM
Well in that case we need to actually calculate the filename and calculate the relative path to the project
There's an extension point that you can call, HopGuiFileOpenedDialog with a HopGuiFileOpenedExtension object passed in.
The tricky part will be to actually calculate that folder for the Internal variable without actually loading the metadata.
hansva
7:57 PM
but you would have to calculate the variable in the destination folder, not the source folder
matt
7:57 PM
Yes
hansva
7:57 PM
and relative paths will also be a fun one to calculate
matt
7:58 PM
Fairly straightforward actually.
hansva
7:58 PM
Imagine someone going above the PROJECT_HOME folder
matt
7:58 PM
So whatever file you are processing, the parent folder is ${Internal.Entry.Current.Folder}
Resolve the file expression with that variable set. Pass that into the extension point.
That automatically inserts PROJECT_HOME if it's possible at all.
