From 92fb9a778c0e0fc0645c8f0265fa733a94545f4c Mon Sep 17 00:00:00 2001
From: Anusha Ranganathan <anusha@cottagelabs.com>
Date: Thu, 6 Jul 2023 07:01:37 +0100
Subject: [PATCH] Updated README.md to reflect v0.3.8 of the code
---
README.md | 240 ++++++++++++++++++-------------
hyrax/lib/tasks/setup_hyrax.rake | 2 +-
2 files changed, 140 insertions(+), 102 deletions(-)
diff --git a/README.md b/README.md
index a2d9311c..7b00484e 100644
--- a/README.md
+++ b/README.md
@@ -55,166 +55,202 @@ There are 2 `docker-compose` files provided in the repository, which build the c
By default, this runs on port 3000 internally in docker.
http://web:3000
-* [Wokers container](https://gitlab.ruhr-uni-bochum.de/FDM/rdm-system/rdms/-/blob/master/docker-compose.yml#L112-L119) runs the background tasks, using [sidekiq](https://github.com/mperham/sidekiq) and redis.
+* [Workers container](https://gitlab.ruhr-uni-bochum.de/FDM/rdm-system/rdms/-/blob/master/docker-compose.yml#L112-L119) runs the background tasks, using [sidekiq](https://github.com/mperham/sidekiq) and redis.
* Hyrax processes long-running or particularly slow work in background jobs to speed up the web request/response cycle. When a user submits a file through a work (using the web or an import task), there a number of background jobs that are run, initilated by the hyrax actor stack, as explained [here](https://samvera.github.io/what-happens-deposit-2.0.html).
You can monitor the background workers using the materials data repository service at http://web:3000/sidekiq when logged in as an admin user.
-### Setup of RDMS at startup
-The RDMS web container is the main entry point of the rdms application, with which users interact.
+### Container volumes
+The data for the application is stored in docker named volumes as specified by the compose files. These are:
-At startup, the web container runs [docker-entrypoint.sh](https://gitlab.ruhr-uni-bochum.de/FDM/rdm-system/rdms/-/blob/master/hyrax/docker-entrypoint.sh). This script does the following tasks
+```bash
+$ docker volume list -f name=rdms
+DRIVER VOLUME NAME
+local rdms_app
+local rdms_cache
+local rdms_db-app
+local rdms_db-fcrepo
+local rdms_derivatives
+local rdms_fcrepo
+local rdms_file_uploads
+local rdms_redis
+local rdms_solr
+```
-#### 1. Initial checks
+These will persist when the system is brought down and rebuilt. Deleting them will require importers etc. to run again.
-* Creates the log folder if it doesn't exist
-* Checks the bundle (and installs It in development)
-* Does the database migration and setup
-* Checks Solr and Fedora are running (waits 15 seconds if needed)
-#### 2. RDMS creates some default users at start-up
+## Running RDMS
-* System users if they don't exist
+* When running in production environment,
- * Sytem administrator - the email id of the user is defined in the env file as `SYSTEM_ADMINISTRATOR`
- * Publication manager - the email id of the user is defined in the env file as `SYSTEM_PUBLICATION_MANAGER`
+ * Prepare your .env file using .env.template as the template.
-* Creates users defined in the file `hyrax/seed/setup.json`, if they haven't already been created.
- **Setting a password for the system users**
+ * You need to use `docker-compose.yml` to build and run the containers.
- The system users are created with a random password. If you need to login as these users, you need to change the password from the rails console (once the web container is up and running)
+ You could setup an alias for docker-compose on your local machine, to ease typing
- ```
- docker exec -it rdms-web-1 /bin/bash
- rails c
- u = User.find_by(email: ENV['SYSTEM_ADMINISTRATOR'])
- u.password = <some password>
- u.save
- ```
+ ```bash
+ alias hd='docker-compose -f docker-compose.yml'
+ ```
- **Creating a set of users during start-up from `setup.json`**
- RDMS uses the file `hyrax/seed/setup.json` to create a set of users during first startup, if the file exists.
+* When running in development and test environment,
- If you would like to create users during startup,
+ * Prepare your .env file using .env.template.development as the template.
- * The file `hyrax/seed/setup.json` needs to exist at build time.
- * Copy the file in `hyrax/seed/setup.json.template` to `hyrax/seed/setup.json`
+ * You need to use `docker-compose.yml` and `docker-compose.development.override.yml`. to build and run your containers.
- * Modify `hyrax/seed/setup.json` so it has the list of users to create / update.
+ You could setup an alias for docker-compose on your local machine, to ease typing
- For more information on the rake task to create users and the json file, see [this wiki](https://gitlab.ruhr-uni-bochum.de/FDM/rdm-system/antleaf-projectmanagement/-/wikis/Development-notes/Create-users-from-the-rails-console) page.
+ ```bash
+ alias hd='docker-compose -f docker-compose.yml -f docker-compose.development.override.yml'
+ ```
- * Build the image
- Note
+* Prepare the file `hyrax/seed/setup.json` if you would like to create a set of users in the RDMS, as a part of start-up.
- If the container is re-started,
+### Build the docker container
- * The task will go through the list of users in `hyrax/seed/setup.json` and check they exists. Their details will not be updated, if setup.json is modified.
+To start with, you would need to build the system, before running the services. To do this you need to issue the `build` command
+```bash
+$ hd build
+```
- * If the file is deleted, it will skip checking the users exist or creating them if they exist.
+### Start and run the services in the containers
- * You do not need to have the file `hyrax/seed/setup.json` at build time to create users at start-up.
+To run the containers after build, issue the `up` command (-d means run as daemon, in the background):
- * The rake task [setup_users.rake](https://gitlab.ruhr-uni-bochum.de/FDM/rdm-system/antleaf-projectmanagement/-/wikis/Development-notes/Create-users-from-the-rails-console) can be used to create users as explained in the wiki page. Once the web container is up,
+```bash
+$ hd up -d
+```
+The containers should all start and the services should be available in their end points as described above
+* web server at http://localhost:3000 in development and https://domain-name in production
- * Execute a shell on the container
- * Prepare a json file with the list of users
- * run the rake task
+### Docker container status and logs
- This is better than having a list of users at build time, as it only needs to be run once.
+You can see the state of the containers with `hd ps`, and view logs e.g. for the web container using `hd logs web`
-#### 3. Loads workflows, create default admin sets and collection types
+The services that you would need to monitor the logs for are docker mainly web and workers.
-* Loads the default workflows
+### Running services
-* Creates the default admin set and collection types
+ * fcrepo container will run the fedora service, which will be available in port 8080 at http://localhost:8080/fcrepo/rest
+ * Solr container will run the Solr service, which will be available in port 8983 at http://localhost:8983/solr
+ * The web container runs the materials data repository service, which will be available in port 3000 at http://localhost:3000
-#### 4. Starts the rails server
+### Using RDMS
+To use the RDMS application on http://localhost:3000, you would need to do the following
-### Container volumes
-The data for the application is stored in docker named volumes as specified by the compose files. These are:
+1. [Add passwords for the system users](#setting-a-password-for-the-system-users), or assign the role admin to a user who has signed in through Shibboleth, or register an user with role admin (see [wiki](https://gitlab.ruhr-uni-bochum.de/FDM/rdm-system/antleaf-projectmanagement/-/wikis/Development-notes/Create-users-from-the-rails-console)), so they can login.
+2. Setup the [RUB publication workflow](https://gitlab.ruhr-uni-bochum.de/FDM/rdm-system/antleaf-projectmanagement/-/wikis/Development-notes/RUB-publication-workflow-v0.3.8), to submit a dataset.
+3. Setup the [CRC 1280 publication workflow](https://gitlab.ruhr-uni-bochum.de/FDM/rdm-system/antleaf-projectmanagement/-/wikis/Development-notes/CRC-1280-publication-workflow-v0.3.8) to submit an experiment.
-```bash
-$ docker volume list -f name=rdms
-DRIVER VOLUME NAME
-local rdms_app
-local rdms_cache
-local rdms_db-app
-local rdms_db-fcrepo
-local rdms_derivatives
-local rdms_fcrepo
-local rdms_file_uploads
-local rdms_redis
-local rdms_solr
-```
+### Stop the services
+You could stop the container using `hd stop`.
-These will persist when the system is brought down and rebuilt. Deleting them will require importers etc. to run again.
+This will just stop all of the running containers created by `hd up`
+### To deploy an update of the code
-## Running in development or test
+similar to the steps described above, to deploy an update of the code
-When running in development and test environment, prepare your .env file using .env.template.development as the template. You need to use `docker-compose -f docker-compose.yml -f docker-compose.development.override.yml`. This will use the docker-compose.yml file and the docker-compose.development.override.yml file.
- * fcrepo container will run the fedora service, which will be available in port 8080 at http://localhost:8080/fcrepo/rest
- * Solr container will run the Solr service, which will be available in port 8983 at http://localhost:8983
- * The web container runs the materials data repository service, which will be available in port 3000 at http://localhost:3000
+ * Checkout the latest code from github
-You could setup an alias for docker-compose on your local machine, to ease typing
+ * Stop the containers. To deploy an update of the code, you likely want to use
-```bash
-alias hd='docker-compose -f docker-compose.yml -f docker-compose.development.override.yml'
-```
+ ```
+ hd down
+ ```
-### Yarn and static assets
+ This will stop containers and remove
-Static asset build is only run in `production` environment to speed up container creation in develop. To see features such as the IIIF viewer, `yarn install` must be run on the web container once it's up.
+ * Containers for services defined in the Compose file
+ * Networks defined in the networks section of the Compose file
+ * The default network, if one is used
-```
-hd run web yarn install
-```
+ Networks and volumes defined as external are not removed. Named volumes are not removed.
-For production envionment the assets precompile may fail. If rdms comes up with an error page this might have happed. Precompile the assets within the web container:
+* Build the system
-get into the web container
-```
-docker exec -it rdms-web-1 /bin/bash
-```
+ ```
+ hd build
+ ```
-From within the web container precompile assets
-```
-bundle exec rake assets:clean assets:precompile
-```
+* Start the containers
-## Builidng, starting and managing the service with docker
+ ```
+ hd up -d
+ ```
-### Build the docker container
+* Check all the containers have started and the status of the web service with the logs
-To start with, you would need to build the system, before running the services. To do this you need to issue the `build` command
-```bash
-$ hd build
-```
+ ```
+ hd ps
+ hd logs -f web
+ ```
-### Start and run the services in the containers
+### Setup of RDMS at startup
-To run the containers after build, issue the `up` command (-d means run as daemon, in the background):
+The RDMS web container is the main entry point of the rdms application, with which users interact.
-```bash
-$ hd up -d
-```
-The containers should all start and the services should be available in their end points as described above
-* web server at http://localhost:3000 in development and https://domain-name in production
+At startup, the web container runs [docker-entrypoint.sh](https://gitlab.ruhr-uni-bochum.de/FDM/rdm-system/rdms/-/blob/master/hyrax/docker-entrypoint.sh). This script does the following tasks
-### docker container status and logs
+#### 1. Initial checks
-You can see the state of the containers with `docker-compose ps`, and view logs e.g. for the web container using `docker-compose logs web`
+* Creates the log folder if it doesn't exist
+* Checks the bundle (and installs It in development)
+* Does the database migration and setup
+* Checks Solr and Fedora are running (waits 15 seconds if needed)
+* Create S3 bucket
-The services that you would need to monitor the logs for are docker mainly web and workers.
+#### 2. System users
+
+RDMS creates system users if they don't exist
+
+* System administrator - the email id of the user is defined in the .env file as `SYSTEM_ADMINISTRATOR`
+* Publication manager - the email id of the user is defined in the .env file as `SYSTEM_PUBLICATION_MANAGER`
+
+ ##### Setting a password for the system users
+
+ The system users are created with a random password. If you need to login as these users, you need to change the password from the rails console (once the web container is up and running)
+
+ ```
+ docker exec -it rdms-web-1 /bin/bash
+ rails c
+ u = User.find_by(email: ENV['SYSTEM_ADMINISTRATOR'])
+ u.password = <some password>
+ u.save
+ ```
+
+#### 3. Loads workflows, create default admin sets and collection types
+
+* Loads the default workflows
+* Creates the default collection types and admin sets (Hyrax administrative task)
+* Setup the participants, visibility and workflow for each admin set
+* Setup of CRC 1280 collection
+
+Creates users defined in the file `hyrax/seed/setup.json`, if they haven't already been created.
+
+#### 4. Create users during start-up from `setup.json`
+RDMS uses the file `hyrax/seed/setup.json` to create a set of users during first startup, if the file exists.
+
+If you would like to create users during startup,
+
+* Copy the file in `hyrax/seed/setup.json.template` to `hyrax/seed/setup.json`
+
+* Modify `hyrax/seed/setup.json` so it has the list of users to create / update.
+
+ For more information on the rake task to create users and the json file, see [this wiki](https://gitlab.ruhr-uni-bochum.de/FDM/rdm-system/antleaf-projectmanagement/-/wikis/Development-notes/Create-users-from-the-rails-console) page.
+
+**Note**: The file `hyrax/seed/setup.json` needs to exist before running docker build, for users to be created at start-up.
+
+#### 4. Starts the rails server
### Some example docker commands and usage:
@@ -249,7 +285,9 @@ docker exec -it rdms-web-1 /bin/bash
## Backups
-There is [docker documentation](https://docs.docker.com/storage/volumes/#backup-restore-or-migrate-data-volumes) advising how to back up volumes and their data.
+There is [docker documentation](https://docs.docker.com/storage/volumes/#backup-restore-or-migrate-data-volumes) advising how to back up volumes and their data. Docker suggests mounting the volumes in a container, creating a tar of the contents of the volume in a backup location and restoring them.
+
+It is also possible to stop the containers, copy all of the named volumes in /var/lib/docker/volumes and start the containers. To use the backup, copy the volumes back into /var/lib/docker/volumes.
diff --git a/hyrax/lib/tasks/setup_hyrax.rake b/hyrax/lib/tasks/setup_hyrax.rake
index 3263bb78..ed6e97a1 100644
--- a/hyrax/lib/tasks/setup_hyrax.rake
+++ b/hyrax/lib/tasks/setup_hyrax.rake
@@ -25,8 +25,8 @@ namespace :rdms do
Rake::Task['hyrax:default_admin_set:create'].invoke
Rake::Task['hyrax:default_admin_set_for_workflow:create'].invoke
# the publication manager has manage role for the workflow, but not approving.
- # So updating, so they also have approving role.
# Approving role can only be assigned after workflow is created, not before.
+ # Re-assigning the role publication manager helps acquire the approving permission.
Rake::Task['rdms:publication_manager:update'].invoke
######
# Finished loading workflows, creating collection types and administrative sets
--
GitLab