model diff and update only if needed?

[zetaab]

Checklist

• I agree to the terms within the OpenFGA Code of Conduct.

Describe the problem you'd like to have solved

Currently it is pretty difficult to manage different versions of models automatically in openfga. For instance, If I add model to my application and always on (kubernetes pod) startup I will "create model" it will create TONS OF different models to database. What I would like to see is that, it could check what is the latest model and apply new model only if needed. Could there be like somekind of mechanism to check is update needed?

It is not really great situation that models needs to be managed manually basically in production environments. Otherwise the database is full of trash.

Describe the ideal solution

I have been thinking way how database migrations are working. Own table which will store the state which version of model is in use and is there newer version available which should be applied.

Is it possible to calculate like checksum for current model and to be applied model? That way we could compare is the model apply needed (perhaps worth of provide functions for that in openfga itself).

Alternatives and current workarounds

No response

Additional context

No response

[aaguiarz]

@ewanharris spiked some work to enhance the CLI to display the differences between two models and return with a specific code depending on the outcome.

@zetaab do you think that would solve the issue for you?

[zetaab]

@aaguiarz if there is api which can return diff between "currently used model" and "model that I would like to apply". Then it is solving my issue. Even boolean value is the current model same than the code is trying to apply. That is enough to provide the information that is needed.

[aaguiarz]

@zetaab what @ewanharris did was not a new API but a CLI feature. You'd need to call the CLI to check if the model changed, and if it did, write the model.

We could even add a CLI parameter that to fga model write that skips writing it and returns the same model_id if the model did not change.

[zetaab]

Well i would not like to use cli. Instead i would like to use go sdk 😆

[aaguiarz]

Would you use the SDK to write the model on kubernetes pod startup?

[zetaab]

yep, I have application which have many replicas. Always when replica start, these will check "is there need for database migrations". It could do similar check against openfga. "Do I need to update the model". I do not want to have anything manual steps in production. Those can lead to issues when someone just forget to run that manual command after some commit is deployed to production

[c-thiel]

+1 for this. Further motivation can be found in openfga/openfga#1132 and openfga/cli#331.

From a developer perspective an experience similar to how rust SQLx or python alembic or other DB Tools / ORMs handle database schema migration would be ideal.

An easy first step in that direction would be to allow store-ids and auth-model-ids to be static information - let the client specify those optionally on creation.

[hebestreit]

@zetaab what @ewanharris did was not a new API but a CLI feature. You'd need to call the CLI to check if the model changed, and if it did, write the model.

We could even add a CLI parameter that to fga model write that skips writing it and returns the same model_id if the model did not change.

Is the cli feature which @ewanharris was working on already implemented? Unfortunately I couldn't find any reference for the spike.

We're currently trying to understand how an update of the model can be implemented inside Kubernetes. Do any of you have any best practices?

[c-thiel]

@hebestreit we are using an automated deployment with our OSS project Lakekeeper that depends on OpenFGA.
To resolve the issue, we store a mapping of auth model id in the application to OpenFGA auth model ID in OpenFGA itself.
This way we can query the store to figure out if an auth model is available.

FGA Yaml

type auth_model_id

type model_version
  relations
    define applied: [auth_model_id]
    define exists: [auth_model_id:*]

You can find most of the logic right around here: https://github.com/lakekeeper/lakekeeper/blob/166f4ca9f026b166da0b81a2962a9da8f75fce57/crates/iceberg-catalog/src/service/authz/implementations/openfga/migration/mod.rs#L34

Hope it helps.

[hebestreit]

@c-thiel thanks a lot for sharing the implementation of lakekeeper which also seems to be a very interesting project. I didn't think about storing the information directly inside OpenFGA and hard-code the target version in the app. 👍

The section Extending Authz in the Developer Guide also helped me to understand it in more details.

[OsmanMElsayed]

+1 for this as well.

We use github actions to deploy the model, but the issue is that the github action gets triggered by a deployment pipeline on a regular basis - i.e. it is not triggered when the model changes are pushed to a branch. Therefore, we end up calling fga model write way too much even if there are not changes in the model.

@aaguiarz I'd also love to know more about the spike in the CLI that you were referring to, may be it will solve our issue 😊.