Introduction

rdms is an application based on Hyrax 3.3 stack by Cottage Labs and AntLeaf. It is built with Docker containers.

Getting Started

Clone the repository with git clone https://gitlab.ruhr-uni-bochum.de/FDM/rdm-system/rdms.git.

Ensure you have docker and docker-compose.

Open a console and try running docker -h and docker-compose -h to verify they are both accessible.

Create the environment file .env. You can start by copying the template file .env.template.development to .env and customizing the values to your setup.
Note: For production environment, use .env.template as your template.

quick start

If you would like to do a test run of the system, start the docker containers

$ cd rdms
$ docker-compose up -d

You should see the containers being built and the services start.

Docker compose explained

There are 2 docker-compose files provided in the repository, which build the containers running the services as shown above

• docker-compose.yml is the main docker-compose file. It builds all the core servcies required to run the application

• docker-compose.override.yml is used along with the main docker-compose.yml file in development, mainly to expose ports for the various services.

Containers running in docker

• fcrepo is the container running the Fedora 4 commons repository, an rdf document store.

  By default, this runs the fedora service on port 8080 internally in docker. http://fcrepo:8080/fcrepo/rest

• Solr container runs SOLR, an enterprise search server.

  By default, this runs the SOLR service on port 8983 internally in docker. http://solr:8983

• db containers running a postgres database for use by the Hyrax application (appdb) and Fedora (fcrepodb).

  By default, this runs the database service on port 5432 internally in docker.

• redis container running redis, used by Hyrax to manage background tasks.

  By default, this runs the redis service on port 6379 internally in docker.

• app container sets up the Hyrax application, which is then used by 2 services - web and workers.

  • Web container runs the application.

    By default, this runs on port 3000 internally in docker. http://web:3000

  This container runs docker-entrypoint.sh, on startup. The bash script

  • creates the log folder

  • checks the bundle (and installs It in development)

  • does the database setup

  • checks Solr and Fedora are running

  • It runs a rake task setup_hyrax.rake to setup the application. This rake task

    • creates users listed in setup.json

    • Loads the default workflows

    • Creates the default admin set and collection types

  • Starts the rails server

  • Wokers container runs the background tasks, using sidekiq and redis.

    Hyrax processes long-running or particularly slow work in background jobs to speed up the web request/response cycle. When a user submits a file through a work (using the web or an import task), there a number of background jobs that are run, initilated by the hyrax actor stack, as explained here.

    You can monitor the background workers using the materials data repository service at http://web:3000/sidekiq when logged in as an admin user.

The data for the application is stored in docker named volumes as specified by the compose files. These are:

$ docker volume list -f name=rdms
DRIVER    VOLUME NAME
local     rdms_app
local     rdms_cache
local     rdms_db
local     rdms_db-fcrepo
local     rdms_derivatives
local     rdms_fcrepo
local     rdms_file_uploads
local     rdms_redis
local     rdms_solr

These will persist when the system is brought down and rebuilt. Deleting them will require importers etc. to run again.

Running in development or test

When running in development and test environment, prepare your .env file using .env.template.development as the template. You need to use docker-compose -f docker-compose.yml -f docker-compose.override.yml. This will use the docker-compose.yml file and the docker-compose.override.yml file.

• fcrepo container will run the fedora service, which will be available in port 8080 at http://localhost:8080/fcrepo/rest
• Solr container will run the Solr service, which will be available in port 8983 at http://localhost:8983
• The web container runs the materials data repository service, which will be available in port 3000 at http://localhost:3000

You could setup an alias for docker-compose on your local machine, to ease typing

alias hd='docker-compose -f docker-compose.yml -f docker-compose.override.yml'

Yarn and static assets

Static asset build is only run in production environment to speed up container creation in develop. To see features such as the IIIF viewer, yarn install must be run on the web container once it's up.

hd run web yarn install

Builidng, starting and managing the service with docker

Build the docker container

To start with, you would need to build the system, before running the services. To do this you need to issue the build command

$ hd build

Start and run the services in the containers

To run the containers after build, issue the up command (-d means run as daemon, in the background):

$ hd up -d

The containers should all start and the services should be available in their end points as described above

• web server at http://localhost:3000 in development and https://domain-name in production

docker container status and logs

You can see the state of the containers with docker-compose ps, and view logs e.g. for the web container using docker-compose logs web

The services that you would need to monitor the logs for are docker mainly web and workers.

Some example docker commands and usage:

Docker cheat sheet

# Bring the whole application up to run in the background, building the containers
hd up -d --build

# Stop the container
hd stop

# Halt the system
hd down

# Re-create the web container without affecting the rest of the system (and run in the background with -d)
hd up -d --build --no-deps --force-recreate web

# View the logs for the web application container
hd logs web

# Create a log dump file
hd logs web | tee web_logs_`date --iso-8601`
# (writes to e.g. web_logs_2022-02-14)

# View all running containers
hd ps     

# Using its container name, you can run a shell in a container to view or make changes directly
docker exec -it rdms-web-1 /bin/bash

Backups

There is docker documentation advising how to back up volumes and their data.