Creating a Wrapd Project - With Gradle
Step 1 - Get the Tools
Make sure you have the required tools for this tutorial:
- Java JDK 11+
- Gradle 7.1+ (older versions are probably fine)
- MySQL 5+. For tutorial purposes we’ll use MySQL, though Wrapd can work with almost any JDBC-compatible DBMS and has automated tests for PostgreSQL, SQLite, MS SQL Server, and MySQL. If this is your first time through this tutorial, it may be easiest to set up a MySQL DBMS using Docker as described below.
(Optional) Use Docker to Run a MySQL DBMS
Install Docker and create a Docker Compose file:
- Install Docker from https://docker.com
- Create a suitable directory and create a file called docker-compose.yml with the following content:
version: '3.9'
services:
mysql-db:
image: mysql:8.0.25
command: --default-authentication-plugin=mysql_native_password
restart: always
volumes:
- db-data-wrapd_mysql_myproject:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: password
MYSQL_DATABASE: wrapd_myproject
MYSQL_USER: user
MYSQL_PASSWORD: password
ports:
- "3306:3306"
volumes:
db-data-wrapd_mysql_myproject:
driver: local
To launch the Docker MySQL instance:
- Change to the directory containing docker-compose.yml
- Run:
docker-compose up -d
To shut down the Docker MySQL instance and retain the database for next time:
- Change to the directory containing docker-compose.yml
- Run:
docker-compose down
To shut down the Docker MySQL instance and delete the database:
- Change to the directory containing docker-compose.yml
- Run:
docker-compose down -v
Step 2 - Start a Gradle Project
Create a directory to host your project. Change to it.
Type gradle init
and press Enter.
- For project type, choose “2: application”.
- For implementation language, choose “3: Java”.
- In response to “Split functionality across multiple subprojects?” choose “2: yes - application and library projects”.
- For the build script DSL, choose “1: Groovy”.
- Pick whatever project name you like.
- For “Source package”, put org.reldb.myproject. You can, of course, use whatever package you like, but for tutorial purposes we’ll use org.reldb.myproject.
Your project skeleton will now be created. Get a directory listing to see what it created. You should see something like this:
drwxr-xr-x 13 dave staff 416 26 Oct 21:40 .
drwxr-xr-x 4 dave staff 128 26 Oct 21:33 ..
-rw-r--r-- 1 dave staff 154 26 Oct 21:40 .gitattributes
-rw-r--r-- 1 dave staff 103 26 Oct 21:40 .gitignore
drwxr-xr-x 6 dave staff 192 26 Oct 21:36 .gradle
drwxr-xr-x 4 dave staff 128 26 Oct 21:40 app
drwxr-xr-x 4 dave staff 128 26 Oct 21:40 buildSrc
drwxr-xr-x 3 dave staff 96 26 Oct 21:36 gradle
-rwxr-xr-x 1 dave staff 8070 26 Oct 21:36 gradlew
-rw-r--r-- 1 dave staff 2763 26 Oct 21:36 gradlew.bat
drwxr-xr-x 4 dave staff 128 26 Oct 21:40 list
-rw-r--r-- 1 dave staff 392 26 Oct 21:40 settings.gradle
drwxr-xr-x 4 dave staff 128 26 Oct 21:40 utilities
If you created a docker-compose.yml file above, you may want to move it to this project directory.
Some things to note:
- As a “multiple subprojects” project, Gradle has created utilities and list as examples of library subprojects and app as the application subproject. We’ll rename them shortly.
- The buildSrc subproject, which often contains imperative custom build logic, can also contain settings common to some of or all the other subprojects.
- The gradle & .gradle directories and gradlew & gradlew.bat files – along with buildSrc – provide Gradle build scripts and infrastructure.
- The settings.gradle file defines top-level settings, such as what subprojects should be built.
- The .gitattributes and .gitignore files contain handy defaults if you’re using git to manage your source code.
Step 3 - Create a Database Subproject
Connecting to a database, even an empty one, is fundamental to Wrapd’s way of working. You need to start with a (possibly empty) database. You can use your own or the Docker-based MySQL DBMS instance described above, but the remainder of this tutorial assumes you’re using the Docker-based MySQL DBMS instance, or at least equivalent settings and credentials with your own database.
First, let’s turn the utilities subproject into a database subproject. It will provide connectivity to a SQL DBMS, and will be used by the other subprojects.
- Rename utilities to database. This renames the subproject directory.
- Edit settings.gradle and change utilities to database. This tells Gradle to build the database subproject.
- Rename database/src/main/java/org/reldb/myproject/utilities to database/src/main/java/org/reldb/myproject/database. This renames the formerly-utilities directory to a more appropriate database.
- Delete the sample auto-generated files in database/src/main/java/org/reldb/myproject/database.
-
Edit app/build.gradle and change “utilities” in the following to “database”:
dependencies { implementation 'org.apache.commons:commons-text' implementation project(':utilities') }
It should now be:
dependencies { implementation 'org.apache.commons:commons-text' implementation project(':database') }
This tells the app subproject to reference the database subproject. The ‘org.apache.commons:commons-text’ dependency won’t be used in this tutorial.
Now try gradle clean build
to verify that the build works so far. You should see BUILD SUCCESSFUL.
Step 4 - Configure the Database Subproject
-
In database/build.gradle, replace the following…
dependencies { api project(':list') }
…with…
dependencies { implementation 'org.reldb:Wrapd:1.0.0' implementation 'mysql:mysql-connector-java:8.0.27' testImplementation 'org.junit.jupiter:junit-jupiter:5.7.2' testImplementation 'org.junit.platform:junit-platform-runner:1.7.2' }
This declares that the database subproject depends on:
- Wrapd 1.0.0
- MySQL Connector 8.0.27
- The JUnit test framework (but only for testing)
-
Put the following in database/src/main/java/org/reldb/myproject/GetDatabase.java, to define the GetDatabase class with a getDatabase() method for obtaining Database instances.
package org.reldb.myproject.database; import com.mysql.cj.jdbc.MysqlDataSource; import org.reldb.wrapd.sqldb.Database; import java.io.InputStream; import java.util.Properties; public class GetDatabase { private static final String PROPERTIES_NAME = "db.properties"; public static Database getDatabase() throws Exception { try (InputStream propertiesSource = GetDatabase.class.getClassLoader().getResourceAsStream(PROPERTIES_NAME)) { var properties = new Properties(); if (propertiesSource == null) throw new Exception("Missing " + PROPERTIES_NAME); properties.load(propertiesSource); var dataSource = new MysqlDataSource(); dataSource.setURL(properties.getProperty("db.url")); dataSource.setUser(properties.getProperty("db.user")); dataSource.setPassword(properties.getProperty("db.password")); var tableNamePrefix = properties.getProperty("db.tablename_prefix", ""); return new Database( dataSource, tableNamePrefix, null ); } } }
-
Put the following in database/src/main/resources/db.properties to define the database settings.
db.tablename_prefix=wrapd_myproject db.url=jdbc:mysql://localhost/wrapd_myproject db.user=user db.password=password
These values work with the Docker MySQL database defined in the docker-compose.yml file at the start of this document. If you’re using your own MySQL DBMS, change as appropriate. If you’re using a MySQL DBMS other than the Docker instance, you will need to create the database beforehand.
-
Put the following in database/src/test/java/org/reldb/myproject/TestGetDatabase.java to test your database connection.
package org.reldb.myproject.database; import org.junit.jupiter.api.Test; import static org.junit.jupiter.api.Assertions.fail; public class TestGetDatabase { @Test void verifyDatabaseConnectionSuccessful() { try { GetDatabase.getDatabase(); } catch (Exception e) { e.printStackTrace(); fail("Unable to connect to database.", e); } } }
- Delete everything in:
- app/src/main/java/org/reldb/myproject/app/
- app/src/test/java/org/reldb/myproject/app/
This is example code that was auto-generated by Gradle and is no longer needed.
-
Launch the Docker MySQL DBMS container, as described at the top of this document.
- Run
gradle clean build
to verify that the build works so far. You should see BUILD SUCCESSFUL.
Step 5 - Create a Schema Subproject
Wrapd contains a simple but effective schema migrator that can be used to automate schema initialisation and upgrades on multiple target databases. This avoids the error-prone complexity of manually managing schema updates.
Well turn the list subproject into a schema subproject. It will create the initial schema for this tutorial, and demonstrate how schema changes can be migrated. Later, we’ll copy it to a queries subproject to become the basis for the main feature of Wrapd: turning tested SQL queries into Java methods to invoke them.
- Rename list to schema. This renames the subproject directory.
- Edit settings.gradle and change list to schema. This tells Gradle to build the schema subproject.
- Rename schema/src/main/java/org/reldb/myproject/list to database/src/main/java/org/reldb/myproject/schema. This renames the formerly-utilities directory to a more appropriate database.
- Delete the sample auto-generated files in schema/src/main/java/org/reldb/myproject/database.
- Delete the directory schema/src/test. We won’t need it for now.
-
Edit schema/build.gradle and add the following after the plugins block:
dependencies { implementation project(':database') implementation 'org.reldb:Wrapd:1.0.0' }
This tells the schema subproject to reference the database subproject because it will need to connect to the database. It’s also dependent on Wrapd.
Now try gradle clean build
to verify that the build works so far. You should see BUILD SUCCESSFUL.
Step 6 - Configure the Schema Subproject
In schema/src/main/java/org/reldb/myproject/schema create a file called Schema.java with the following content:
package org.reldb.myproject.schema;
import org.reldb.toolbox.progress.ConsoleProgressIndicator;
import org.reldb.wrapd.response.Response;
import org.reldb.wrapd.response.Result;
import org.reldb.wrapd.schema.SQLSchema;
import org.reldb.wrapd.sqldb.Database;
import org.reldb.myproject.database.GetDatabase;
public class Schema extends SQLSchema {
public Schema(Database database) {
super(database);
}
@Override
protected Update[] getUpdates() {
return new Update[] {
schema -> {
getDatabase().updateAll("CREATE TABLE $$tester01 (x INT NOT NULL PRIMARY KEY, y INT NOT NULL)");
return Result.OK;
},
};
}
public static void main(String[] args) throws Exception {
var schema = new Schema(GetDatabase.getDatabase());
var result = schema.setup(new ConsoleProgressIndicator());
if (result.isOk())
System.out.println("OK: Schema has been set up.");
else
Response.printError("ERROR in Schema: Schema creation:", result.error);
}
}
Now add the following to the end of schema/gradle.build:
task runSchemaSetup(type: JavaExec) {
group = "Wrapd"
description "Ensure that the schema is up-to-date."
classpath = sourceSets.main.runtimeClasspath
mainClass = "org.reldb.myproject.Schema"
}
That adds a Gradle task called ‘runSchemaSetup’ that will run the Schema main(…) method to generate the schema. It can be run as often as you like, as it will only build a new schema if needed.
Run gradle clean build
to verify that the build works so far. You should see BUILD SUCCESSFUL.
Now make sure your MySQL DBMS instance is running and run gradle runSchemaSetup
to generate the initial schema. You should see something like:
...
> Task :schema:runSchemaSetup
Creating schema: 0.0% complete.
Schema created: 50.0% complete.
Updating to version 1: 50.0% complete.
Updated to version 1: 100.0% complete.
OK: Schema has been set up.
BUILD SUCCESSFUL in 1s
...
If you run the gradle runSchemaSetup
task again, you should see something like:
...
> Task :schema:runSchemaSetup
OK: Schema has been set up.
...
This indicates that it recognised the schema was up-to-date, and didn’t need to perform any updates.
Later, we’ll add to the schema definition to see how Wrapd automates schema migration. For now, we have a database with a table named wrapd_myprojecttester01 with integer columns x and y.
Step 7 - Create the Query Generator
The main purpose of Wrapd is to turn straightforward SQL query definitions into invocable Java methods. We’ll make a copy of the schema subproject for that purpose.
Simply copy everything in schema to a new subproject called query. Then:
- Edit settings.gradle to reference the new query subproject. It should look like this:
rootProject.name = 'MyProject' include('app', 'schema', 'database', 'query')
- Rename query/main/java/org/reldb/myproject/schema to query/main/java/org/reldb/myproject/query.
- Delete any files in query/main/java/org/reldb/myproject/query. We’re going to replace them.
- Create a file called Definitions.java in query/main/java/org/reldb/myproject/query with the following content:
package org.reldb.myproject.query; import org.reldb.toolbox.utilities.Directory; import org.reldb.wrapd.sqldb.Database; import org.reldb.wrapd.sqldb.Definer; import org.reldb.myproject.database.GetDatabase; public class Definitions extends Definer { public Definitions(Database database, String codeDirectory, String packageSpec) { super(database, codeDirectory, packageSpec); } void generate() throws Throwable { purgeTarget(); defineTable("$$tester01"); defineQuery("SelectTester", "SELECT * FROM $$tester01 WHERE x = {xValue}", 1); defineUpdate("ClearTester", "DELETE FROM $$tester01"); emitDatabaseAbstractionLayer("DatabaseAbstractionLayer"); } public static void main(String[] args) throws Throwable { var db = GetDatabase.getDatabase(); var codeDirectory = "../app/src/main/java"; var codePackage = "org.reldb.myproject.app.generated"; if (!Directory.chkmkdir(codeDirectory)) { System.out.println("ERROR creating code directory " + codeDirectory); return; } var sqlDefinitions = new Definitions(db, codeDirectory, codePackage); sqlDefinitions.generate(); System.out.println("OK: Queries are ready."); } }
- Edit query/gradle.build to change this:
task runSchemaSetup(type: JavaExec) { group = "Wrapd" description "Ensure that the schema is up-to-date." classpath = sourceSets.main.runtimeClasspath mainClass = "org.reldb.myproject.schema.Schema" }
…to this, which will create a runQueryGenerate Gradle task to generate Java code from the SQL query definitions:
task runQueryGenerate(type: JavaExec) { group = "Wrapd" description "Generate database abstraction layer." classpath = sourceSets.main.runtimeClasspath mainClass = "org.reldb.myproject.query.Definitions" }
- Run
gradle clean build
to verify that the build works so far. You should see BUILD SUCCESSFUL. - Run
gradle runQueryGenerate
task to generate Java code. You should see output similar to the following:... > Task :query:runQueryGenerate Target ../app/src/main/java/org/reldb/myproject/app/generated has been purged. OK: Queries are ready. ...
Now take a look in the app/src/main/java/org/reldb/myproject/app/generated directory to see the code generated by Wrapd and the runQueryGenerate task.
Step 8 - Create the Application
In Step 7, we created a Query subproject that converts your query definitions into Java code to execute them. We ran it to verify that it works. The result is the generated code now available in your app subproject, waiting to be used. Let’s use it.
- In app/build.gradle replace implementation ‘org.apache.commons:commons-text’ with implementation ‘org.reldb:Wrapd:1.0.0’ to use Wrapd instead of Apache commons-text.
- In app/src/main/java/org/reldb/myproject/app create a file called App.java with the following contents:
package org.reldb.myproject.app; import org.reldb.myproject.app.generated.*; import org.reldb.myproject.database.GetDatabase; public class App { public static void main(String args[]) throws Exception { var database = GetDatabase.getDatabase(); var dbAbstraction = new DatabaseAbstractionLayer(database); // Clear table dbAbstraction.clearTester(); // Populate table for (int x = 0; x < 100; x++) { var row = new Tester01Tuple(database); row.x = x; row.y = x * 10 + 2; row.insert(); } // Show table dbAbstraction.tester01().forEach(System.out::println); // Show a row dbAbstraction.selectTester(2).forEach(System.out::println); } }
- Run
gradle clean build
to verify that the build works so far. You should see BUILD SUCCESSFUL. - Run
gradle run
to run the demonstration application. You should see it emit the query results.
This demonstrates the basic process for creating a Wrapd application. In the following steps, we’ll handle schema migration and demonstrate more queries.
Step 9 - Make a Schema Change
Now we’ll make a schema change, to demonstrate how schema migration is handled.
Remember that we defined the initial database schema in schema/src/main/java/org/reldb/myproject/schema/Schema in Step 5 and Step 6. Now we’ll modify it.
The schema is currently defined by:
@Override
protected Update[] getUpdates() {
return new Update[] {
schema -> {
getDatabase().updateAll("CREATE TABLE $$tester01 (x INT NOT NULL PRIMARY KEY, y INT NOT NULL)");
return Result.OK;
},
};
}
The getUpdates() method returns an array of Update, where each Update defines a schema migration. The first array entry defines the initial database schema. We add migrations by simply adding them to the array.
So, for example, if we want to add another table we can change the above to this:
@Override
protected Update[] getUpdates() {
return new Update[] {
schema -> {
getDatabase().updateAll("CREATE TABLE $$tester01 (x INT NOT NULL PRIMARY KEY, y INT NOT NULL)");
return Result.OK;
},
schema -> {
getDatabase().updateAll("CREATE TABLE $$tester02 (p VARCHAR(20) NOT NULL PRIMARY KEY, q FLOAT NOT NULL, r INT)");
return Result.OK;
},
};
}
Then run gradle clean build
and verify that it emits BUILD SUCCESSFUL.
Now run gradle runSchemaSetup
. You should see the following output:
...
> Task :schema:runSchemaSetup
Updating to version 2: 0.0% complete.
Updated to version 2: 100.0% complete.
OK: Schema has been set up.
BUILD SUCCESSFUL in 1s
...
This shows that the schema has been successfully migrated. The current version is maintained within the database, so it will work on any connectable database to correctly migrating it from its current state or version (including empty) to the latest version.
Migration can either be invoked within the application (such as on every startup) to ensure that any database to which it connects is automatically migrated, or migration can be deployed as a separate application to migrate databases outside the application.
In this tutorial it’s invoked as a main method, to be run as a standalone application:
public static void main(String[] args) throws Exception {
var schema = new Schema(GetDatabase.getDatabase());
var result = schema.setup(new ConsoleProgressIndicator());
if (result.isOk())
System.out.println("OK: Schema has been set up.");
else
Response.printError("ERROR in Schema: Schema creation:", result.error);
}
It can just as easily be invoked as a method to be run from an application startup:
public static Result migrate throws Exception {
var schema = new Schema(GetDatabase.getDatabase());
return schema.setup(new ConsoleProgressIndicator());
}
...
var migrationResult = migrate();
if (result.isOk())
// ...proceed with startup...
else
// ...fail with notification of result.error...
...
Being able to revert migrations with specified regression steps will be a feature of a future Wrapd release.
Note that schema migrations must always – and only – be added as a new array entry to be returned by getUpdates(). Update queries must never be added to previous array entries and schema migrations must never be done outside of this mechanism or chaos will ensue.
Step 10 - More (on) Queries
In the query definitions shown in Step 7 (see query/src/main/java/org/reldb/myproject/query/Definitions.java), we saw three kinds of query definition:
...
void generate() throws Throwable {
purgeTarget();
defineTable("$$tester01");
defineQuery("SelectTester", "SELECT * FROM $$tester01 WHERE x = {xValue}", 1);
defineUpdate("ClearTester", "DELETE FROM $$tester01");
emitDatabaseAbstractionLayer("DatabaseAbstractionLayer");
}
...
defineTable(“$tester01”);
This is the simplest definition, which generates the following methods in the file specified by emitDatabaseAbstractionLayer(…):
public Stream<Tester01Tuple> tester01() throws SQLException {
return Tester01.query(database);
}
public Stream<Tester01Tuple> tester01(java.sql.Connection connection) throws SQLException {
return Tester01.query(database, connection);
}
public Stream<Tester01Tuple> tester01ForUpdate(java.sql.Connection connection) throws SQLException {
return Tester01.queryForUpdate(database, connection);
}
public Stream<Tester01Tuple> tester01ForUpdate() throws SQLException {
return Tester01.queryForUpdate(database);
}
Note that these methods are wrappers around static methods in a generated class called Tester01, the same name as the database table. If you choose not to invoke emitDatabaseAbstractionLayer(…), the Tester01 class can be referenced directly.
The first two tester01(…) methods return all rows and columns in the table – by invoking “SELECT * FROM table” – and return a Stream of Tester01Tuple. Tester01Tuple is automatically generated. The second method accepts a parameter of type Connection, typically used when wrapped by a transaction.
The last two tester01ForUpdate(…) methods also return all rows and columns in the table, but each Tester01Tuple instance has been prepared for subsequent invocation of its update(…) method. If update(…) is invoked on the results of a non-Update tester01(…) method, an exception will be thrown.
defineQuery(“SelectTester”, “SELECT * FROM $$tester01 WHERE x = {xValue}”, 1);
The definition above generates the following methods in the file specified by emitDatabaseAbstractionLayer(…):
public Stream<SelectTesterTuple> selectTester(java.sql.Connection connection, java.lang.Integer xValue) throws SQLException {
return SelectTester.query(database, connection, xValue);
}
public Stream<SelectTesterTuple> selectTester(java.lang.Integer xValue) throws SQLException {
return SelectTester.query(database, xValue);
}
These methods are wrappers around static methods in a generated class called SelectTester, the name specified in the first parameter of defineQuery(…). If you choose not to invoke emitDatabaseAbstractionLayer(…), the SelectTester class can be referenced directly.
Note that the definition specifies a parameter {xValue} and a corresponding argument 1. The argument value doesn’t matter – the query doesn’t have to return results. Only its type matters, and is used to specify the corresponding Java parameter type of the generated methods, and will have the same name as the parameter specified in the query.
defineUpdate(“ClearTester”, “DELETE FROM $$tester01”);
This definition generates the following methods in the file specified by emitDatabaseAbstractionLayer(…):
public boolean clearTester(java.sql.Connection connection) throws SQLException {
return ClearTester.update(database, connection);
}
public boolean clearTester() throws SQLException {
return ClearTester.update(database);
}
These methods are wrappers around static methods in a generated class called ClearTester, the name specified in the first parameter of defineQuery(…). If you choose not to invoke emitDatabaseAbstractionLayer(…), the ClearTester class can be referenced directly.
The methods invoke the definition’s DELETE query, and return boolean true if the query returns a result set, and false otherwise. Under normal circumstances, the query method should always return false. In typical use, the return value is ignored; query failure is indicated by throwing an exception.
WARNING WARNING WARNING WARNING WARNING WARNING
defineUpdate(…) will execute the specified query during code generation! DO NOT USE THIS ON A PRODUCTION DATABASE!!! A future Wrapd update will provide a ‘safety’ switch to prevent inadvertent execution of such update queries.
WARNING WARNING WARNING WARNING WARNING WARNING
Running Queries in a Transaction
The generated query methods can be invoked within a database transaction.
For example, the tutorial application invokes the following methods, in app/src/main/java/org/reldb/myproject/app/App.java:
// Clear table
dbAbstraction.clearTester();
// Populate table
for (int x = 0; x < 100; x++) {
var row = new Tester01Tuple(database);
row.x = x;
row.y = x * 10 + 2;
row.insert();
}
// Show table
dbAbstraction.tester01().forEach(System.out::println);
// Show a row
dbAbstraction.selectTester(2).forEach(System.out::println);
We can also invoke them in a database transaction as follows:
// Do the above in a transaction...
database.processTransaction(xact -> {
dbAbstraction.clearTester(xact);
for (int x = 0; x < 100; x++) {
var row = new Tester01Tuple(database);
row.x = x;
row.y = x * 10 + 2;
row.insert(xact);
}
dbAbstraction.tester01(xact).forEach(System.out::println);
dbAbstraction.selectTester(xact, 2).forEach(System.out::println);
return Result.OK;
});
The transaction only commits if the lambda returns any successful Response, such as Result.OK or Response.set(0), etc. Anything else – an error Response or an exception thrown – causes the transaction to be aborted, i.e., rolled back.
The result of processTransaction is whatever Response is passed back as a return value from the transaction lambda.
Other Query Definitions
There are other query definitions we haven’t seen yet.
defineTable(String tableName, String whereClause, Object… args)
This is a variation on the defineTable(…) method used above. The method used above is designed to return all rows in the table. This method defines a query method that returns a set of rows specified by a SQL WHERE clause. For example:
defineTable("$$MyTable", "a >= {minLimit} AND a < {maxLimit}", 1, 2);
It generates a method called myTable that returns the rows specified by the SQL expression passed as the second argument.
This example is parametric, so it will generate methods with two parameters, minLimit and maxLimit.
defineValueOf(String queryName, String sqlText, Object… args)
This is a variation on the defineQuery(…) method used above.
This method returns the value in the first column of the first row of the query result. It returns that value as an Optional