Encryption and already not encrypted databases

[Sesa1988]

Hi,

I want to encrypt the database file but many of my users have already their databases not encrypted.

Is there a way to check if the database file is encrypted and if not call "encrypt"?

This way I could avoid a database migration what seams very dangerous to do locally.

[alextekartik]

Unfortunately no, you can just try to open a file with a codec (using encryption or not), it will fail if it does not match.

However the safest is to use a different file name for the regular and encrypted one (and delete the regular one when the encrypted database is created) and check if the encrypted database exists - DatabaseFactory.databaseExists() - first to find out whether you have an encrypted database.

[Sesa1988]

Unfortunately no, you can just try to open a file with a codec (using encryption or not), it will fail if it does not match.

However the safest is to use a different file name for the regular and encrypted one (and delete the regular one when the encrypted database is created) and check if the encrypted database exists - DatabaseFactory.databaseExists() - first to find out whether you have an encrypted database.

Yes but thats very risky and could lead to data loss in the worst case.

I think it would be a cool feature if you configure encryption it will also open an not encrypted database and write it back encrypted.

Thats how I would expect it to work.

[alextekartik]

I think it would be a cool feature if you configure encryption it will also open an not encrypted database and write it back encrypted.

As you said that's risky and that might take a long time the first time the database is opened. I'll look into that to see if this is feasible without too much impact and risk

[Sesa1988]

I think it would be a cool feature if you configure encryption it will also open an not encrypted database and write it back encrypted.

As you said that's risky and that might take a long time the first time the database is opened. I'll look into that to see if this is feasible without too much impact and risk

Its probably less risky and more of a default flow as deleting a database with real data and creating a new one or not?

I hope its possible :)

[alextekartik]

So I tried to...but that was getting too messy to handle 2 input format and having a mixed usage of encrypted and unencrypted data so I gave up.

However I added (somehow missing) DatabaseFactory.databaseExists() in sembast 3.7.4 and added documentation on how to perform a migration from a non encrypted to an encrypted database:

Encrypting an existing database

It is unfortunately not possible to encrypt an existing database in place as a database format is tight to the codec
used. You have to create a new one and copy the data. You can use databaseMerge to copy the data
in a single operation/transaction.

Below is an example of how to convert an existing non-encrypted database to an encrypted one:

// Encryption codec to use (to set as you wish)
SembastCodec? encryptionCodec;

// Existing db name, to convert and remove if it exists
var nonEncryptedDbName = 'my_database.db';
var encryptedDbName = 'my_database_encrypted.db';

late Database db;
// Check if non encrypted database exists
if (await factory.databaseExists(nonEncryptedDbName)) {
  // This should only happen once but could be a lengthy operation

  // Open the non encrypted database
  var nonEncryptedDb = await factory.openDatabase(nonEncryptedDbName);

  // Create a new encrypted database and copy the existing content
  await factory.deleteDatabase(encryptedDbName);
  db = await factory.openDatabase(encryptedDbName,
    codec: encryptionCodec);
  // Copy the content
  await databaseMerge(db, sourceDatabase: nonEncryptedDb);

  // Close and delete the non encrypted database
  await nonEncryptedDb.close();
  await factory.deleteDatabase(nonEncryptedDbName);
} else {
// Open the encrypted database
db = await factory.openDatabase(encryptedDbName,
    codec: encryptionCodec);
}

[Sesa1988]

So I tried to...but that was getting too messy to handle 2 input format and having a mixed usage of encrypted and unencrypted data so I gave up.

However I added (somehow missing) DatabaseFactory.databaseExists() in sembast 3.7.4 and added documentation on how to perform a migration from a non encrypted to an encrypted database:

Encrypting an existing database

It is unfortunately not possible to encrypt an existing database in place as a database format is tight to the codec
used. You have to create a new one and copy the data. You can use databaseMerge to copy the data
in a single operation/transaction.

Below is an example of how to convert an existing non-encrypted database to an encrypted one:

// Encryption codec to use (to set as you wish)
SembastCodec? encryptionCodec;

// Existing db name, to convert and remove if it exists
var nonEncryptedDbName = 'my_database.db';
var encryptedDbName = 'my_database_encrypted.db';

late Database db;
// Check if non encrypted database exists
if (await factory.databaseExists(nonEncryptedDbName)) {
// This should only happen once but could be a lengthy operation

  // Open the non encrypted database
  var nonEncryptedDb = await factory.openDatabase(nonEncryptedDbName);

  // Create a new encrypted database and copy the existing content
  await factory.deleteDatabase(encryptedDbName);
  db = await factory.openDatabase(encryptedDbName,
    codec: encryptionCodec);
  // Copy the content
  await databaseMerge(db, sourceDatabase: nonEncryptedDb);

  // Close and delete the non encrypted database
  await nonEncryptedDb.close();
  await factory.deleteDatabase(nonEncryptedDbName);
} else {
// Open the encrypted database
db = await factory.openDatabase(encryptedDbName,
    codec: encryptionCodec);
}

Looks good thanks!

Will databaseExists open the database and prevent me from replacing it and reading the replaced content? Its for another usecase. I used the File-Type and appDir for exist checks.

[alextekartik]

Will databaseExists open the database

No it just checks whether the file exists

[Sesa1988]

Will databaseExists open the database

No it just checks whether the file exists

I am pretty sure "reload" modifies the lastModifed date of the database file. Its not an issue for me because I have to modify it myself for my usecase anyway and worked around that but I just let you know.

Btw is it possible to merge that reload feature in the version that does not require the latest Flutter version? There is a bug with local auth and pop child routes on the latest version that prevents my release.

[alextekartik]

I am pretty sure "reload" modifies the lastModifed date of the database file.

reload() might trigger a compact operation that will indeed modify the modified data but in general it should not.

Btw is it possible to merge that reload feature in the version that does not require the latest Flutter version?

Hard for me to test/spend time on that, sorry. One solution for you is to only force the latest sembast version in dependency_overrides and leave the main dependency to sembast: any

[Sesa1988]

I am pretty sure "reload" modifies the lastModifed date of the database file.

reload() might trigger a compact operation that will indeed modify the modified data but in general it should not.

Btw is it possible to merge that reload feature in the version that does not require the latest Flutter version?

Hard for me to test/spend time on that, sorry. One solution for you is to only force the latest sembast version in dependency_overrides and leave the main dependency to sembast: any

Np! Thanks for the reload feature. I will try the overwrite or will try to fork it until Google fixes the local auth bug. Thx!