Uploaded image for project: 'OFBiz'
  1. OFBiz
  2. OFBIZ-9501

Move all data in applications to the datamodel component

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: Upcoming Release
    • Fix Version/s: None
    • Component/s: ALL APPLICATIONS
    • Labels:

      Description

      As per a community discussion thread we decided to move all data (seed, seed-initial, demo) to the datamodel component. and to also reduce and cleanup this data to ease debugging and maintenance.

      After moving all the data, the directory and file structure would be similar to the below:
      data/
      ├── demo
      │ ├── AccountingDemoData.xml
      │ ├── CommonDemoData.xml
      │ ├── ContentDemoData.xml
      │ ├── HumanresDemo.xml
      │ ├── ManufacturingDemoData.xml
      │ ├── MarketingDemoData.xml
      │ ├── OrderDemoData.xml
      │ ├── PartyDemoData.xml
      │ ├── ProductDemoData.xml
      │ ├── ShipmentDemoData.xml
      │ └── WorkEffortDemoData.xml
      ├── seed
      │ ├── AccountingSeedData.xml
      │ ├── CommonSeedData.xml
      │ ├── ContentSeedData.xml
      │ ├── HumanresSeedData.xml
      │ ├── ManufacturingSeedData.xml
      │ ├── MarketingSeedData.xml
      │ ├── OrderSeedData.xml
      │ ├── PartySeedData.xml
      │ ├── ProductSeedData.xml
      │ ├── ShipmentSeedData.xml
      │ └── WorkEffortSeedData.xml
      └── seed-initial
      ├── AccountingSeedInitialData.xml
      ├── CommonSeedInitialData.xml
      ├── ContentSeedInitialData.xml
      ├── HumanresSeedInitialData.xml
      ├── ManufacturingSeedInitialData.xml
      ├── MarketingSeedInitialData.xml
      ├── OrderSeedInitialData.xml
      ├── PartySeedInitialData.xml
      ├── ProductSeedInitialData.xml
      ├── ShipmentSeedInitialData.xml
      └── WorkEffortSeedInitialData.xml

      1. OFBIZ-9051-004.patch
        71 kB
        sourabh jain
      2. OFBIZ-9501_Consolidated.patch
        394 kB
        Rishi Solanki
      3. OFBIZ-9501-001.patch
        25 kB
        Taher Alkhateeb
      4. OFBIZ-9501-002.patch
        14 kB
        Prakhar Kumar
      5. OFBIZ-9501-003.patch
        25 kB
        Saurabh Dixit
      6. OFBIZ-9501-WorkEffortSeedData.patch
        85 kB
        Prakhar Kumar

        Activity

        Hide
        devanshu.vyas Devanshu Vyas added a comment -

        Hello Taher,

        Looking at your plan for naming the files, I feel that a suffix 'Data' should be added for all the data files.

        Show
        devanshu.vyas Devanshu Vyas added a comment - Hello Taher, Looking at your plan for naming the files, I feel that a suffix 'Data' should be added for all the data files.
        Hide
        taher Taher Alkhateeb added a comment -

        Yeah, what you're saying makes sense. We can add the suffix to all data files. Maybe we can also use CamelCase instead of hyphenated-names to stay consistent with the rest of the framework. Good idea!

        Show
        taher Taher Alkhateeb added a comment - Yeah, what you're saying makes sense. We can add the suffix to all data files. Maybe we can also use CamelCase instead of hyphenated-names to stay consistent with the rest of the framework. Good idea!
        Hide
        taher Taher Alkhateeb added a comment -

        I've updated the filenames in the description to reflect the camel case and data suffix

        Show
        taher Taher Alkhateeb added a comment - I've updated the filenames in the description to reflect the camel case and data suffix
        Hide
        taher Taher Alkhateeb added a comment -

        After analyzing some of the data (I will get the first commit soon) I believe we should not move any security or portlet data. These things are specific to widgets and applications, not the data model. The only data we should move is proper modeling data

        Show
        taher Taher Alkhateeb added a comment - After analyzing some of the data (I will get the first commit soon) I believe we should not move any security or portlet data. These things are specific to widgets and applications, not the data model. The only data we should move is proper modeling data
        Hide
        taher Taher Alkhateeb added a comment -

        Attaching first patch for review. I tried to keep it small. There are too many ContactMechTypePurpose entities. I'm not sure if we should move some of them to demo instead of seed? Opinions appreciated.

        Also, looking forward to you guys taking on this project

        Show
        taher Taher Alkhateeb added a comment - Attaching first patch for review. I tried to keep it small. There are too many ContactMechTypePurpose entities. I'm not sure if we should move some of them to demo instead of seed? Opinions appreciated. Also, looking forward to you guys taking on this project
        Hide
        swapnilmmane Swapnil M Mane added a comment -

        Thank you Taher Alkhateeb for the patch. This will be helpful

        For ContactMechTypePurpose, IMO it should be the part of seed data because it is the very rich defined list, applicable to most of the businesses in the world.
        By keeping it in seed data, it will be available to all the users and they will not have overhead to copy-paste this for their custom development.

        Show
        swapnilmmane Swapnil M Mane added a comment - Thank you Taher Alkhateeb for the patch. This will be helpful For ContactMechTypePurpose, IMO it should be the part of seed data because it is the very rich defined list, applicable to most of the businesses in the world. By keeping it in seed data, it will be available to all the users and they will not have overhead to copy-paste this for their custom development.
        Hide
        taher Taher Alkhateeb added a comment -

        Thank you Swapnil, commit is applied in r1802603. I forgot to add the seed file in ofbiz-component so I took care of that before committing. All tests pass. Ok, one commit down, maybe another 100 to go

        Show
        taher Taher Alkhateeb added a comment - Thank you Swapnil, commit is applied in r1802603. I forgot to add the seed file in ofbiz-component so I took care of that before committing. All tests pass. Ok, one commit down, maybe another 100 to go
        Hide
        swapnilmmane Swapnil M Mane added a comment -

        Thank you Taher Alkhateeb!

        Show
        swapnilmmane Swapnil M Mane added a comment - Thank you Taher Alkhateeb !
        Hide
        rishisolankii Rishi Solanki added a comment -

        Just as additional information so that sync would be easy to go, here is sequence needs to be follow;

        <component-loader xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                xsi:noNamespaceSchemaLocation="http://ofbiz.apache.org/dtds/component-loader.xsd">
            <load-component component-location="datamodel"/>
            <load-component component-location="party"/>
            <load-component component-location="securityext"/>
            <load-component component-location="content"/>
            <load-component component-location="workeffort"/>
            <load-component component-location="product"/>
            <load-component component-location="manufacturing"/>
            <load-component component-location="accounting"/>
            <load-component component-location="humanres"/>
            <load-component component-location="order"/>
            <load-component component-location="marketing"/>
        <!-- common component used by most other components last because it uses info from most components-->
            <load-component component-location="commonext"/>
        </component-loader>
        
        
        Show
        rishisolankii Rishi Solanki added a comment - Just as additional information so that sync would be easy to go, here is sequence needs to be follow; <component-loader xmlns:xsi= "http: //www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation= "http: //ofbiz.apache.org/dtds/component-loader.xsd" > <load-component component-location= "datamodel" /> <load-component component-location= "party" /> <load-component component-location= "securityext" /> <load-component component-location= "content" /> <load-component component-location= "workeffort" /> <load-component component-location= "product" /> <load-component component-location= "manufacturing" /> <load-component component-location= "accounting" /> <load-component component-location= "humanres" /> <load-component component-location= "order" /> <load-component component-location= "marketing" /> <!-- common component used by most other components last because it uses info from most components--> <load-component component-location= "commonext" /> </component-loader>
        Hide
        swapnilmmane Swapnil M Mane added a comment -

        Thank you Rishi Solanki, this will be helpful!

        Show
        swapnilmmane Swapnil M Mane added a comment - Thank you Rishi Solanki , this will be helpful!
        Hide
        taher Taher Alkhateeb added a comment -

        Thank you for sharing the info rishi, that should come in handy. I suppose the component loading sequence might be helpful but we should not stick to it literally because what matters in the end is that all data loads succesfully.

        Feel free to commit / submit patches folks. I don't have anything in the pipeline for now.

        Show
        taher Taher Alkhateeb added a comment - Thank you for sharing the info rishi, that should come in handy. I suppose the component loading sequence might be helpful but we should not stick to it literally because what matters in the end is that all data loads succesfully. Feel free to commit / submit patches folks. I don't have anything in the pipeline for now.
        Hide
        taher Taher Alkhateeb added a comment - - edited

        Hi Prakhar,

        Thank you for your work, its really great to see people contributing to this JIRA.

        I have a question: Aren't the below records from your patch belonging to the "project" component? If yes wouldn't it be better to move it to that component instead? I'm not sure if there are also more records, that was just a quick skim.

        <WorkEffortContentType workEffortContentTypeId="PROJECT_DISCUSSION" description="Project Discussion" parentTypeId=""/>
        <WorkEffortContentType workEffortContentTypeId="PROJECT_UPDATE" description="Project Update" parentTypeId=""/>
        <WorkEffortContentType workEffortContentTypeId="PROJECT_SPEC" description="Project Specification" parentTypeId=""/>
        
        <WorkEffortPurposeType description="Project" workEffortPurposeTypeId="WEPT_PROJECT"/>
        <WorkEffortPurposeType description="Phase" workEffortPurposeTypeId="WEPT_PHASE"/>
        
        <WorkEffortType description="Workflow Process" hasTable="N" parentTypeId="" workEffortTypeId="WORK_FLOW"/>
        <WorkEffortType description="Workflow Activity" hasTable="N" parentTypeId="" workEffortTypeId="ACTIVITY"/>
        <WorkEffortType description="Task" hasTable="N" parentTypeId="" workEffortTypeId="TASK"/>
        <WorkEffortType description="Milestone" hasTable="N" parentTypeId="" workEffortTypeId="MILESTONE"/>
        <WorkEffortType description="Event" hasTable="N" parentTypeId="" workEffortTypeId="EVENT"/>
        <WorkEffortType description="Available" hasTable="N" parentTypeId="" workEffortTypeId="AVAILABLE"/>
        <WorkEffortType description="Program" hasTable="N" parentTypeId="" workEffortTypeId="PROGRAM"/>
        <WorkEffortType description="Project" hasTable="N" parentTypeId="" workEffortTypeId="PROJECT"/>
        <WorkEffortType description="Project Phase" hasTable="N" parentTypeId="" workEffortTypeId="PHASE"/>
        
        
        Show
        taher Taher Alkhateeb added a comment - - edited Hi Prakhar, Thank you for your work, its really great to see people contributing to this JIRA. I have a question: Aren't the below records from your patch belonging to the "project" component? If yes wouldn't it be better to move it to that component instead? I'm not sure if there are also more records, that was just a quick skim. <WorkEffortContentType workEffortContentTypeId= "PROJECT_DISCUSSION" description= "Project Discussion" parentTypeId=""/> <WorkEffortContentType workEffortContentTypeId= "PROJECT_UPDATE" description= "Project Update" parentTypeId=""/> <WorkEffortContentType workEffortContentTypeId= "PROJECT_SPEC" description= "Project Specification" parentTypeId=""/> <WorkEffortPurposeType description= "Project" workEffortPurposeTypeId= "WEPT_PROJECT" /> <WorkEffortPurposeType description= "Phase" workEffortPurposeTypeId= "WEPT_PHASE" /> <WorkEffortType description= "Workflow Process" hasTable= "N" parentTypeId= "" workEffortTypeId=" WORK_FLOW"/> <WorkEffortType description= "Workflow Activity" hasTable= "N" parentTypeId= "" workEffortTypeId=" ACTIVITY"/> <WorkEffortType description= "Task" hasTable= "N" parentTypeId= "" workEffortTypeId=" TASK"/> <WorkEffortType description= "Milestone" hasTable= "N" parentTypeId= "" workEffortTypeId=" MILESTONE"/> <WorkEffortType description= "Event" hasTable= "N" parentTypeId= "" workEffortTypeId=" EVENT"/> <WorkEffortType description= "Available" hasTable= "N" parentTypeId= "" workEffortTypeId=" AVAILABLE"/> <WorkEffortType description= "Program" hasTable= "N" parentTypeId= "" workEffortTypeId=" PROGRAM"/> <WorkEffortType description= "Project" hasTable= "N" parentTypeId= "" workEffortTypeId=" PROJECT"/> <WorkEffortType description= "Project Phase" hasTable= "N" parentTypeId= "" workEffortTypeId=" PHASE"/>
        Hide
        rishisolankii Rishi Solanki added a comment -

        Taher Alkhateeb Thanks for your review and input. Agree with your point, we should try to move this data to project component. Prakhar Kumar could you please remove the data Taher suggested and upload the patch again. After that check for the possibility and dependencies to move same in the Project component. And submit separate patch for that. Thanks!

        Show
        rishisolankii Rishi Solanki added a comment - Taher Alkhateeb Thanks for your review and input. Agree with your point, we should try to move this data to project component. Prakhar Kumar could you please remove the data Taher suggested and upload the patch again. After that check for the possibility and dependencies to move same in the Project component. And submit separate patch for that. Thanks!
        Hide
        saurabh.dixit Saurabh Dixit added a comment - - edited

        Provided patch to move data from content component to datamodel component. In the patch we have moved the data which only belongs to content. For now leaving other data in the file.

        Also I noticed that DataResource and ElectronicText exists in the seed data. On exploring I came to know that the data is for rendering the subcontent. Please suggest if we should consider it as demo or it is fine to have such data in seed/type data file.
        Reference: OFBIZ-9501-003.patch

        Show
        saurabh.dixit Saurabh Dixit added a comment - - edited Provided patch to move data from content component to datamodel component. In the patch we have moved the data which only belongs to content. For now leaving other data in the file. Also I noticed that DataResource and ElectronicText exists in the seed data. On exploring I came to know that the data is for rendering the subcontent. Please suggest if we should consider it as demo or it is fine to have such data in seed/type data file. Reference: OFBIZ-9501 -003.patch
        Hide
        sourabh.jain sourabh jain added a comment -

        Moved product type data from product component to datamodel component. Attached fourth patch in the sequence for review.

        Show
        sourabh.jain sourabh jain added a comment - Moved product type data from product component to datamodel component. Attached fourth patch in the sequence for review.
        Hide
        prakhar.kumar Prakhar Kumar added a comment -

        Thanks Taher Alkhateeb and Rishi Solanki for reviewing the patch and providing inputs. I have updated the patch with the suggested changes. I will check for the possibility and dependencies to move same in the Project component and will submit a separate patch for that.

        Show
        prakhar.kumar Prakhar Kumar added a comment - Thanks Taher Alkhateeb and Rishi Solanki for reviewing the patch and providing inputs. I have updated the patch with the suggested changes. I will check for the possibility and dependencies to move same in the Project component and will submit a separate patch for that.
        Hide
        rishisolankii Rishi Solanki added a comment -

        Taher Alkhateeb / All,

        While working and reviewing the work of moving some seed data to data model component. I observe few entities data is in all components. Below are few names;

        • StatusType
        • StatusItem
        • StatusValidChange
        • EnumerationType
        • Enumeration
        • CustomMethodType
        • CustomMethod

        Also observe some data which may belongs to other component and exists as seed data in other component. Below are few entity names wrt Party component data exists in other components;

        • RoleType
        • ContactMechType
        • ContactMechPurposeType
        • ContactMechTypePurpose

        My question is how should we handle this while moving the data. I could think of two options;
        1) Move the data as is, that means if any data exists in the Party component then we should move that data into PartySeedData.xml (for now considering only seed data and with mindset data must be loaded successfully). Later we can think if some more logical movements required Or may think to keep them as is so that component bifurcation per seed (any) data remains intact.
        2) Move the data to relevant component file, for example in the above example we would move all role types in the PartySeedData.xml and in data comment we could mention the role types used for which component.

        Please share you input on this, if we decide this then we will be able to move the data quickly.

        Show
        rishisolankii Rishi Solanki added a comment - Taher Alkhateeb / All, While working and reviewing the work of moving some seed data to data model component. I observe few entities data is in all components. Below are few names; StatusType StatusItem StatusValidChange EnumerationType Enumeration CustomMethodType CustomMethod Also observe some data which may belongs to other component and exists as seed data in other component. Below are few entity names wrt Party component data exists in other components; RoleType ContactMechType ContactMechPurposeType ContactMechTypePurpose My question is how should we handle this while moving the data. I could think of two options; 1) Move the data as is, that means if any data exists in the Party component then we should move that data into PartySeedData.xml (for now considering only seed data and with mindset data must be loaded successfully). Later we can think if some more logical movements required Or may think to keep them as is so that component bifurcation per seed (any) data remains intact. 2) Move the data to relevant component file, for example in the above example we would move all role types in the PartySeedData.xml and in data comment we could mention the role types used for which component. Please share you input on this, if we decide this then we will be able to move the data quickly.
        Hide
        taher Taher Alkhateeb added a comment -

        Hi Rishi, I'm not sure but on first impression I think maybe this data should go into Common<reader>Data.xml file. This way we nicely layout fundamental data before higher order data. I'm not sure if this breaks anything though

        Show
        taher Taher Alkhateeb added a comment - Hi Rishi, I'm not sure but on first impression I think maybe this data should go into Common<reader>Data.xml file. This way we nicely layout fundamental data before higher order data. I'm not sure if this breaks anything though
        Hide
        swapnilmmane Swapnil M Mane added a comment -

        Thanks Rishi Solanki for sharing your exploration, and Taher Alkhateeb for your comment.

        I personally like the #1 proposal, since this will improved the readability of data file and will create a logical separation.

        Here is my thought process for this,
        for e.g. StatusItem is different for Order and Content, so it will be nice to have them in separate OrderSeedData.xml and ContentSeedData.xml

        <StatusItem description="Approved" sequenceId="05" statusCode="APPROVED" statusId="ORDER_APPROVED" statusTypeId="ORDER_STATUS"/>
        
        <StatusItem description="Final Draft" sequenceId="04" statusCode="FINAL_DRAFT" statusId="CTNT_FINAL_DRAFT" statusTypeId="CONTENT_STATUS"/>
        

        Thanks!

        Show
        swapnilmmane Swapnil M Mane added a comment - Thanks Rishi Solanki for sharing your exploration, and Taher Alkhateeb for your comment. I personally like the #1 proposal, since this will improved the readability of data file and will create a logical separation. Here is my thought process for this, for e.g. StatusItem is different for Order and Content, so it will be nice to have them in separate OrderSeedData.xml and ContentSeedData.xml <StatusItem description= "Approved" sequenceId= "05" statusCode= "APPROVED" statusId= "ORDER_APPROVED" statusTypeId= "ORDER_STATUS" /> <StatusItem description= "Final Draft" sequenceId= "04" statusCode= "FINAL_DRAFT" statusId= "CTNT_FINAL_DRAFT" statusTypeId= "CONTENT_STATUS" /> Thanks!
        Hide
        taher Taher Alkhateeb added a comment -

        What Swapnil is saying makes sense. On second thought I agree with him

        Show
        taher Taher Alkhateeb added a comment - What Swapnil is saying makes sense. On second thought I agree with him
        Hide
        rishisolankii Rishi Solanki added a comment -

        Thanks Taher and Swapnil for your comment. My preference is same, will go with #1.

        Show
        rishisolankii Rishi Solanki added a comment - Thanks Taher and Swapnil for your comment. My preference is same, will go with #1.
        Hide
        taher Taher Alkhateeb added a comment -

        Hi Rishi and everyone,

        Congratulations again on getting committer ranks. I got a little lost now in keeping track of which patch to commit. Are you interested in taking over the coordination of this JIRA? I'll try to help out as much as I can if you're interested in taking the lead on this.

        Show
        taher Taher Alkhateeb added a comment - Hi Rishi and everyone, Congratulations again on getting committer ranks. I got a little lost now in keeping track of which patch to commit. Are you interested in taking over the coordination of this JIRA? I'll try to help out as much as I can if you're interested in taking the lead on this.
        Hide
        rishisolankii Rishi Solanki added a comment -

        Taher Alkhateeb: Thank you again

        Only the patch attached by you is in the repository all other patches needs to be taken care. This is in my list, but I just wanted to put all seed data for one component in one patch. The patch will include all the conclusion Swapnil, you and me draw. Very soon (by this week) team will attach new patch for single component and will start committing the changes.

        Thanks again for your help on this.

        Show
        rishisolankii Rishi Solanki added a comment - Taher Alkhateeb : Thank you again Only the patch attached by you is in the repository all other patches needs to be taken care. This is in my list, but I just wanted to put all seed data for one component in one patch. The patch will include all the conclusion Swapnil, you and me draw. Very soon (by this week) team will attach new patch for single component and will start committing the changes. Thanks again for your help on this.
        Hide
        rishisolankii Rishi Solanki added a comment -

        Committed changes in revisions 1806258, 1806265 and 1806267. Thanks Saurabh Dixit Saurabh Jain and Prakhar Kumar for providing patch.

        Thanks Taher Alkhateeb Swapnil M Mane for help and review.

        Show
        rishisolankii Rishi Solanki added a comment - Committed changes in revisions 1806258, 1806265 and 1806267. Thanks Saurabh Dixit Saurabh Jain and Prakhar Kumar for providing patch. Thanks Taher Alkhateeb Swapnil M Mane for help and review.
        Hide
        prakhar.kumar Prakhar Kumar added a comment - - edited

        Provided the patch to move data from work effort component to data model component. In this patch, I have moved only the seed data. Also, I have moved some of the dependent party and content seed data to their respective files in data model component.

        Reference: OFBIZ-9501-WorkEffortSeedData.patch

        Show
        prakhar.kumar Prakhar Kumar added a comment - - edited Provided the patch to move data from work effort component to data model component. In this patch, I have moved only the seed data. Also, I have moved some of the dependent party and content seed data to their respective files in data model component. Reference: OFBIZ-9501-WorkEffortSeedData.patch
        Hide
        rishisolankii Rishi Solanki added a comment -

        Here is the consolidated patch covers seed and seed-initial data for party, security ext, content and work effort component. We have leave few data as is in the component and think over them later where can we move that data.

        Thanks to sourabh jain Saurabh Dixit Prakhar Kumar for your patches.

        I would like to commit the changes in this patch.

        Show
        rishisolankii Rishi Solanki added a comment - Here is the consolidated patch covers seed and seed-initial data for party, security ext, content and work effort component. We have leave few data as is in the component and think over them later where can we move that data. Thanks to sourabh jain Saurabh Dixit Prakhar Kumar for your patches. I would like to commit the changes in this patch.
        Hide
        taher Taher Alkhateeb added a comment -

        Hello Rishi and everyone,

        I see this patch is substantially large, which means a good amount of effort went into it. Thank you!

        My recommendation is to go ahead and commit this. I took a quick glance and things seem to be in proper order, but it would be lengthy and difficult to review a patch of this size, and since multiple people worked on this I trust we're in good hands.

        Well done folks, keep them coming!

        Show
        taher Taher Alkhateeb added a comment - Hello Rishi and everyone, I see this patch is substantially large, which means a good amount of effort went into it. Thank you! My recommendation is to go ahead and commit this. I took a quick glance and things seem to be in proper order, but it would be lengthy and difficult to review a patch of this size, and since multiple people worked on this I trust we're in good hands. Well done folks, keep them coming!
        Hide
        rishisolankii Rishi Solanki added a comment -

        Taher,
        Thanks for your continuous support. I'll commit it by today once tests pass.

        Show
        rishisolankii Rishi Solanki added a comment - Taher, Thanks for your continuous support. I'll commit it by today once tests pass.
        Hide
        jacques.le.roux Jacques Le Roux added a comment -

        Thanks for this remarkable effort Rishi!

        Show
        jacques.le.roux Jacques Le Roux added a comment - Thanks for this remarkable effort Rishi!

          People

          • Assignee:
            rishisolankii Rishi Solanki
            Reporter:
            taher Taher Alkhateeb
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:

              Development