More docs, tests for advanced migrations

docs/content/en/docs/Advanced Features/migrations.md

@@ -49,13 +49,106 @@ can be used together with `customStatement` to run the statements.
 ## Complex migrations
 
 Sqlite has builtin statements for simple changes, like adding columns or dropping entire tables.
+More complex migrations require a [12-step procedure](https://www.sqlite.org/lang_altertable.html#otheralter) that
+involes creating a copy of the table and copying over data from the old table.
+Moor 3.4 introduced the `TableMigration` api to automate most of this procedure, making it easier and safer to use.
 
-Complex migrations require a [12-step procedure](https://www.sqlite.org/lang_altertable.html#otheralter) that involes creating a copy
+To start the migration, moor will create a new instance of the table with the current schema. Next, it will copy over
-of the table and copying over data from the old table.
+rows from the old table.
-Moor 3.4 introduced the `TableMigration` api to automate most of this procedure.
+In most cases, for instance when changing column types, we can't just copy over each row without changing its content.
+Here, you can use a `columnTransformer` to apply a per-row transformation.
+The `columnTransformer` is a map from columns to the sql expression that will be used to copy the column from the
+old table.
+For instance, if we wanted to cast a column before copying it, we could use:
 
-When you're using complex migrations in your app, we strongly recommend to write integration tests for them to avoid
+```dart
-data loss.
+columnTransformer: {
+  todos.category: todos.category.cast<int>(),
+}
+```
+
+Internally, moor will use a `INSERT INTO SELECT` statement to copy old data. In this case, it would look like
+`INSERT INTO temporary_todos_copy SELECT id, title, content, CAST(category AS INT) FROM todos`.
+As you can see, moor will use the expression from the `columnTransformer` map and fall back to just copying the column
+otherwise.   
+If you're introducing new columns in a table migration, be sure to include them in the `newColumns` parameter of
+`TableMigration`. Moor will ensure that those columns have a default value or a transformation in `columnTransformer`.
+Of course, moor won't attempt to copy `newColumns` from the old table either.
+
+Regardless of whether you're implementing complex migrations with `TableMigration` or by running a custom sequence
+of statements, we strongly recommend to write integration tests covering your migrations. This helps to avoid data
+loss caused by errors in a migration.
+
+Here are some examples demonstrating common usages of the table migration api:
+
+### Changing the type of a column
+
+Let's say the `category` column in `Todos` used to be a non-nullable `text()` column that we're now changing to a
+nullable int. For simplicity, we assume that `category` always contained integers, they were just stored in a text
+column that we now want to adapt.
+
+```patch
+class Todos extends Table {
+  IntColumn get id => integer().autoIncrement()();
+  TextColumn get title => text().withLength(min: 6, max: 10)();
+  TextColumn get content => text().named('body')();
+-  IntColumn get category => text()();
++  IntColumn get category => integer().nullable()();
+}
+```
+
+After re-running your build and incrementing the schema version, you can write a migration:
+
+```dart
+onUpgrade: (m, old, to) async {
+  if (old <= yourOldVersion) {
+    await m.alterTable(
+      TableMigration(
+        todos,
+        columnTransformer: {
+          todos.category: todos.category.cast<int>(),
+        }
+      ),
+    );
+  }
+}
+```
+
+The important part here is the `columnTransformer` - a map from columns to expressions that will
+be used to copy the old data. The values in that map refer to the old table, so we can use
+`todos.category.cast<int>()` to copy old rows and transform their `category`.
+All columns that aren't present in `columnTransformer` will be copied from the old table without
+any transformation.
+
+### Changing column constraints
+
+When you're changing columns constraints in a way that's compatible to existing data (e.g. changing
+non-nullable columns to nullable columns), you can just copy over data without applying any
+transformation:
+
+```dart
+await m.alterTable(TableMigration(todos));
+```
+
+### Renaming columns
+
+If you're renaming a column in Dart, note that the easiest way is to just rename the getter and use
+`named`: `TextColumn newName => text().named('old_name')()`. That is fully backwards compatible and
+doesn't require a migration.
+
+If you do want to change the actual column name in a table, you can write a `columnTransformer` to
+use an old column with a different name:
+
+```dart
+await m.alterTable(
+  TableMigration(
+    yourTable, 
+    columnTransformer: {
+      yourTable.newColumn: const CustomExpression('old_column_name')
+    },
+  )
+)
+```
 
 ## Post-migration callbacks
 

docs/content/en/docs/Getting started/advanced_dart_tables.md

@@ -5,7 +5,7 @@ weight: 150
 ---
 
 {{% pageinfo %}}
-__Prefer sql?__: If you prefer, you can also declare tables via `CREATE TABLE` statements.
+__Prefer sql?__ If you prefer, you can also declare tables via `CREATE TABLE` statements.
 Moor's sql analyzer will generate matching Dart code. [Details]({{< ref "starting_with_sql.md" >}}).
 {{% /pageinfo %}}
 
@@ -49,6 +49,12 @@ The updated class would be generated as `CREATE TABLE categories (parent INTEGER
 To update the name of a column when serializing data to json, annotate the getter with 
 [`@JsonKey`](https://pub.dev/documentation/moor/latest/moor/JsonKey-class.html).
 
+You can change the name of the generated data class too. By default, moor will stip a trailing
+`s` from the table name (so a `Users` table would have a `User` data class).
+That doesn't work in all cases though. With the `EnabledCategories` class from above, we'd get
+a `EnabledCategorie` data class. In those cases, you can use the [`@DataClassName`](https://pub.dev/documentation/moor/latest/moor/DataClassName-class.html)
+annotation to set the desired name.
+
 ## Nullability
 
 By default, columns may not contain null values. When you forgot to set a value in an insert,
@@ -138,4 +144,24 @@ Note that the mapping for `boolean`, `dateTime` and type converters only applies
 the database.
 They don't affect JSON serialization at all. For instance, `boolean` values are expected as `true` or `false`
 in the `fromJson` factory, even though they would be saved as `0` or `1` in the database.
-If you want a custom mapping for JSON, you need to provide your own [`ValueSerializer`](https://pub.dev/documentation/moor/latest/moor/ValueSerializer-class.html).
+If you want a custom mapping for JSON, you need to provide your own [`ValueSerializer`](https://pub.dev/documentation/moor/latest/moor/ValueSerializer-class.html).
+
+## Custom constraints
+
+Some column and table constraints aren't supported through moor's Dart api. This includes `REFERENCES` clauses on columns, which you can set
+through `customConstraint`:
+
+```dart
+class GroupMemberships extends Table {
+  IntColumn get group => integer().customConstraint('NOT NULL REFERENCES groups (id)')();
+  IntColumn get user => integer().customConstraint('NOT NULL REFERENCES users (id)')();
+
+  @override
+  Set<Column> get primaryKey => {group, user};
+}
+```
+
+Applying a `customConstraint` will override all other constraints that would be included by default. In
+particular, that means that we need to also include the `NOT NULL` constraint again.
+
+You can also add table-wide constraints by overriding the `customConstraints` getter in your table class.

moor/test/integration_tests/migrations_integration_test.dart

@@ -1,6 +1,6 @@
 @TestOn('vm')
 import 'package:moor/ffi.dart';
-import 'package:moor/moor.dart';
+import 'package:moor/moor.dart' hide isNull;
 import 'package:test/test.dart';
 
 import '../data/tables/todos.dart';
@@ -30,8 +30,7 @@ void main() {
           TableMigration(
             db.todosTable,
             columnTransformer: {
-              db.todosTable.category:
+              db.todosTable.category: db.todosTable.category.cast<int>(),
-                  const CustomExpression('CAST(category AS INT)'),
             },
           ),
         );
@@ -48,4 +47,49 @@ void main() {
     final item = await db.select(db.todosTable).getSingle();
     expect(item.category, 12);
   });
+
+  test('rename columns', () async {
+    // Create todos table with category as category_old
+    final executor = VmDatabase.memory(setup: (db) {
+      db.execute('''
+        CREATE TABLE todos (
+          id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
+          title TEXT NOT NULL,
+          content TEXT NOT NULL,
+          target_date INTEGER NOT NULL,
+          category_old INTEGER NULL
+        );
+      ''');
+
+      db.execute('INSERT INTO todos (title, content, target_date) '
+          "VALUES ('title', 'content', 0)");
+    });
+
+    final db = TodoDb(executor);
+    db.migration = MigrationStrategy(
+      onCreate: (m) async {
+        await m.alterTable(
+          TableMigration(
+            db.todosTable,
+            columnTransformer: {
+              db.todosTable.category: const CustomExpression('category_old'),
+            },
+          ),
+        );
+      },
+    );
+
+    final createStmt = await db
+        .customSelect("SELECT sql FROM sqlite_schema WHERE name = 'todos'")
+        .map((row) => row.readString('sql'))
+        .getSingle();
+
+    expect(
+      createStmt,
+      allOf(contains('category INT'), isNot(contains('category_old'))),
+    );
+
+    final item = await db.select(db.todosTable).getSingle();
+    expect(item.category, isNull);
+  });
 }