Uploaded image for project: 'OFBiz'
  1. OFBiz
  2. OFBIZ-9902

Create a DATAMODELCHANGES.md to track data model changes

    Details

    • Type: Bug
    • Status: In Progress
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: Trunk
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      The file will have the format:

      Entity Changes:
      Here we will have a bulleted list with entity names and the revision changes took place
      1. Added Entities
      2. Removed Entities

      Field changes:
      Here we will show data in tabular form in format
      entityname | field | action | isPK | revision

        Activity

        Hide
        mbrohl Michael Brohl added a comment -

        Thanks for the initiative, Aditya Sharma!

        I think we should additionally provide SQL migration scripts along with the changes.
        As an example: we lately introduced the fromDate as part of the PK in SecurityPermission.

        So additionally to the change log you proposed, I would add a migration SQL for the Derby database.

        ALTER TABLE SECURITY_PERMISSION ADD FROM_DATE DATETIME;
        UPDATE SECURITY_PERMISSION SET FROM_DATE = '2017-01-01 00:00:00.0';
        ALTER TABLE SECURITY_PERMISSION DROP PRIMARY_KEY, ADD PRIMARY KEY (PERMISSION_ID, FROM_DATE);
        

        (this might be syntactically incorrect, it's only an example).

        I also would like to see every change in order of it's introduction. So the order would not be on the entity level but on the database level.
        By changing primary keys and such, relations could also be affected and I think it would make sense to have everything in order, especially for migrations.

        In our db changelogs, we also list new seed data as OFBiz load data to prevent missing data which is needed by added or changed funtionality.

        In my view, the main use case of a db changelog is to help users keep their database up-to-date without the need to monitor everything themselves and do a migration by trial-and-error so it should be as comprehensive as possible.

        As a general rule, I would like to introduce the guideline to provide a changelog along with every database or functionality change affecting the data, latest within the commit. If you take the time to work this out, we should take care to keep it up-to-date in the community.

        Thanks,
        Michael

        Show
        mbrohl Michael Brohl added a comment - Thanks for the initiative, Aditya Sharma ! I think we should additionally provide SQL migration scripts along with the changes. As an example: we lately introduced the fromDate as part of the PK in SecurityPermission. So additionally to the change log you proposed, I would add a migration SQL for the Derby database. ALTER TABLE SECURITY_PERMISSION ADD FROM_DATE DATETIME; UPDATE SECURITY_PERMISSION SET FROM_DATE = '2017-01-01 00:00:00.0'; ALTER TABLE SECURITY_PERMISSION DROP PRIMARY_KEY, ADD PRIMARY KEY (PERMISSION_ID, FROM_DATE); (this might be syntactically incorrect, it's only an example). I also would like to see every change in order of it's introduction. So the order would not be on the entity level but on the database level. By changing primary keys and such, relations could also be affected and I think it would make sense to have everything in order, especially for migrations. In our db changelogs, we also list new seed data as OFBiz load data to prevent missing data which is needed by added or changed funtionality. In my view, the main use case of a db changelog is to help users keep their database up-to-date without the need to monitor everything themselves and do a migration by trial-and-error so it should be as comprehensive as possible. As a general rule, I would like to introduce the guideline to provide a changelog along with every database or functionality change affecting the data, latest within the commit. If you take the time to work this out, we should take care to keep it up-to-date in the community. Thanks, Michael
        Hide
        aditya.sharma Aditya Sharma added a comment -

        Hi Michael,
        Totally agreed on that.
        I am attaching the file with format & some initial data. Please have a look and provide inputs if any.

        Show
        aditya.sharma Aditya Sharma added a comment - Hi Michael, Totally agreed on that. I am attaching the file with format & some initial data. Please have a look and provide inputs if any.
        Hide
        jacques.le.roux Jacques Le Roux added a comment -

        Initially the project privileged a migration service creation rather than a SQL script. I believe a SQL script is acceptable for simple cases like above, but when it's more complicated a migration service should still be privileged.

        Show
        jacques.le.roux Jacques Le Roux added a comment - Initially the project privileged a migration service creation rather than a SQL script. I believe a SQL script is acceptable for simple cases like above, but when it's more complicated a migration service should still be privileged.
        Hide
        mbrohl Michael Brohl added a comment -

        Have we ever provided such a service? How does ist work?
        Some database changes cannot be done by a service because the database has to be changed and migrated BEFORE OFBiz can be startet.

        I doubt that there will be changes which require a service for database migration, everything can be done on the database level with simple SQL scripts and in my opinion, they fit much better in a database changelog than services.

        Show
        mbrohl Michael Brohl added a comment - Have we ever provided such a service? How does ist work? Some database changes cannot be done by a service because the database has to be changed and migrated BEFORE OFBiz can be startet. I doubt that there will be changes which require a service for database migration, everything can be done on the database level with simple SQL scripts and in my opinion, they fit much better in a database changelog than services.
        Show
        jacques.le.roux Jacques Le Roux added a comment - There are some (old) examples in https://cwiki.apache.org/confluence/display/OFBIZ/Revisions+Requiring+Data+Migration+-+upgrade+ofbiz
        Hide
        mbrohl Michael Brohl added a comment -

        There are some SQL migration scripts also so I don't see a clear priviligation of services of SQL?

        The contents I see there are matching the idea of the changelog very well. It's in historial order so that a user can "replay" all changes as provided in the list in correct order. I agree with Sharan's comment there that we should provide not only the data and commit of the change but also the release(s).

        So if we would take this structure, add the missing informations and put it in the document Aditya provided, we would have a good start.

        Show
        mbrohl Michael Brohl added a comment - There are some SQL migration scripts also so I don't see a clear priviligation of services of SQL? The contents I see there are matching the idea of the changelog very well. It's in historial order so that a user can "replay" all changes as provided in the list in correct order. I agree with Sharan's comment there that we should provide not only the data and commit of the change but also the release(s). So if we would take this structure, add the missing informations and put it in the document Aditya provided, we would have a good start.
        Hide
        jacques.le.roux Jacques Le Roux added a comment -

        It's when you deprecate an entity, and "especially when changing the primary key", see https://cwiki.apache.org/confluence/display/OFBIZ/General+Entity+Overview#GeneralEntityOverview-DeprecatedEntities

        Show
        jacques.le.roux Jacques Le Roux added a comment - It's when you deprecate an entity, and "especially when changing the primary key", see https://cwiki.apache.org/confluence/display/OFBIZ/General+Entity+Overview#GeneralEntityOverview-DeprecatedEntities

          People

          • Assignee:
            aditya.sharma Aditya Sharma
            Reporter:
            aditya.sharma Aditya Sharma
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:

              Development