MVC: Example for migrations?

[fraenki]

Hi,

how do I handle (complex) data migrations for plugins and core components? Can someone provide an example?

For example, I have the following model:

Code: [Select]

<model>
    <mount>//OPNsense/FooBar</mount>
    <items>
        <dummies>
            <dummy type="ArrayField">
                <value_a type="OptionField">
                    <OptionValues>
                        <option1>Option No. 1</option1>
                        <option2>Option No. 2</option2>
                        <option3>Option No. 3</option3>
                    </OptionValues>
                </value_a>
                <field1 type="TextField">
                    <Required>N</Required>
                </field1>
                <field2 type="TextField">
                    <Required>N</Required>
                </field2>
            </dummy>
        </dummies>
    </items>
</model>
And I want to migrate it to be more like this:

Code: [Select]

<model>
    <mount>//OPNsense/FooBar</mount>
    <items>
        <dummies>
            <dummy type="ArrayField">
                <value_a type="OptionField">
                    <OptionValues>
                        <option1>Option No. 1</option1>
                        <option2>Option No. 2</option2>
                        <option3>Option No. 3</option3>
                    </OptionValues>
                </value_a>
                <option1_field1 type="TextField">
                    <Required>N</Required>
                </option1_field1>
                <option2_field1 type="TextField">
                    <Required>N</Required>
                </option2_field1>
                <option2_field2 type="TextField">
                    <Required>N</Required>
                </option2_field2>
                <option3_field1 type="TextField">
                    <Required>N</Required>
                </option3_field1>
                <option3_field2 type="TextField">
                    <Required>N</Required>
                </option3_field2>
                <option3_field3 type="TextField">
                    <Required>N</Required>
                </option3_field3>
            </dummy>
        </dummies>
    </items>
</model>
What will be changed? Instead of two multi-purpose fields "field1" and "field2" the updated model will have separate fields for each available option.

In my book the data migration of the config.xml would work something like this:

Code: [Select]

FOREACH dummy
    IF value_a == 'option1'
      # option1 expects only input in "field1", drop other field
      RENAME field1 TO option1_field1
      DELETE field2
    ELIF value_a == 'option2'
      # option2 expects input in both fields
      RENAME field1 TO option2_field1
      RENAME field2 TO option2_field2
    ELIF value_a == 'option3'
      # option3: introduce a 3rd field
      RENAME field1 TO option3_field1
      RENAME field2 TO option3_field2
      CREATE option3_field3 WITH default_value
    END
END
How would I do this in the OPNsense MVC framework? I've seen migration files in opnsense/core, but none of them performs any complex task like this. I'd be grateful for some hints or a working example. Thanks!

Regards
- Frank

[AdSchellevis]

Hi Frank,

You can add your own code in migrations, our unittests contain some samples, like https://github.com/opnsense/core/blob/master/src/opnsense/mvc/tests/app/models/OPNsense/Base/BaseModel/Migrations/M0_0_1.php

In theory you can migrate any structure, you just have to be careful to remove the old data when all data is aligned in the new structure.
It's always a good idea to prepare unittests for you code as well if the migrations turn out to be more complex.


Best regards,

Ad

[fraenki]

Thanks, Ad!
The unit test looks promising, will test it soon.


Regards
- Frank