Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found
Select Git revision

Target

Select target project
  • researchdata/rdms
1 result
Select Git revision
Show changes
Commits on Source (57)
Showing
with 324 additions and 117 deletions
......@@ -177,9 +177,9 @@ The services that you would need to monitor the logs for are docker mainly web a
To use the RDMS application on http://localhost:3000, you would need to do the following
1. [Add passwords for the system users](#setting-a-password-for-the-system-users), or assign the role admin to a user who has signed in through Shibboleth, or register an user with role admin (see [wiki](https://gitlab.ruhr-uni-bochum.de/FDM/rdm-system/antleaf-projectmanagement/-/wikis/Development-notes/Create-users-from-the-rails-console)), so they can login.
2. Setup the [RUB publication workflow](https://gitlab.ruhr-uni-bochum.de/FDM/rdm-system/antleaf-projectmanagement/-/wikis/Development-notes/RUB-publication-workflow-v0.3.8), to submit a dataset.
3. Setup the [CRC 1280 publication workflow](https://gitlab.ruhr-uni-bochum.de/FDM/rdm-system/antleaf-projectmanagement/-/wikis/Development-notes/CRC-1280-publication-workflow-v0.3.8) to submit an experiment.
1. [Add passwords for the system users](#setting-a-password-for-the-system-users), or assign the role admin to a user who has signed in through Shibboleth, or register an user with role admin (see [wiki](https://gitlab.ruhr-uni-bochum.de/researchdata/rdms/-/wikis/system/ReSeeD-user-management)), so they can login.
2. Setup the [RUB publication workflow](https://gitlab.ruhr-uni-bochum.de/researchdata/rdms/-/wikis/system/development-notes/RUB-Publication-Workflow-v0.3.8), to submit a dataset.
3. Setup the [CRC 1280 publication workflow](https://gitlab.ruhr-uni-bochum.de/researchdata/rdms/-/wikis/system/development-notes/CRC1280-Publication-Workflow-v0.3.8) to submit an experiment.
### Stop the services
You could stop the container using `hd stop`.
......@@ -285,7 +285,7 @@ If you would like to create users during startup,
* Modify `hyrax/seed/setup.json` so it has the list of users to create / update.
For more information on the rake task to create users and the json file, see [this wiki](https://gitlab.ruhr-uni-bochum.de/FDM/rdm-system/antleaf-projectmanagement/-/wikis/Development-notes/Create-users-from-the-rails-console) page.
For more information on the rake task to create users and the json file, see [this wiki](https://gitlab.ruhr-uni-bochum.de/researchdata/rdms/-/wikis/system/ReSeeD-user-management) page.
**Note**: The file `hyrax/seed/setup.json` needs to exist before running docker build, for users to be created at start-up.
......
......@@ -87,7 +87,7 @@ module Hyrax
parent_object = file_set.parent_object
siblings = parent_object.file_sets
duplicate_record = siblings.select {|fs| fs.title[0].downcase == file_name && fs.id != file_set.id }
duplicate_record = siblings.select {|fs| fs.title[0]&.downcase == file_name && fs.id != file_set.id }
new_title = nil
if duplicate_record.present?
......@@ -165,7 +165,8 @@ module Hyrax
s3.init_client
source_object_key = uploaded_file.file.path
target_bucket_name = s3.sanitise_name(file_set.parent_works.first.id)
parent_work = file_set.parent_works.first
target_bucket_name = s3.sanitise_name(parent_work.id)
target_bucket = Aws::S3::Bucket.new(target_bucket_name)
......
......@@ -2,21 +2,23 @@ $(document).on('ready', function() {
if ($('#agreementModal').length > 0){
$('#agreementModal').modal('show');
}
let scrollTimeout;
$('#agreementModal .modal-body').on('scroll', function() {
action_buttons = $('#agreementModal .modal-footer > a')
if ($(this)[0].scrollTop + $(this)[0].clientHeight >= $(this)[0].scrollHeight) {
// Enable the buttons when user reaches the bottom
action_buttons.attr('disabled', false);
action_buttons.each(function(){
$(this).attr('href', $(this).data('url'))
})
}
else{
action_buttons.attr('disabled', 'disabled');
action_buttons.attr('href', "javascript:void(0)")
}
clearTimeout(scrollTimeout);
scrollTimeout = setTimeout(() => {
if (($(this)[0].scrollTop + $(this)[0].clientHeight + 10) >= $(this)[0].scrollHeight) {
// Enable the buttons when user reaches the bottom
action_buttons.attr('disabled', false);
action_buttons.each(function(){
$(this).attr('href', $(this).data('url'))
})
}
else{
action_buttons.attr('disabled', 'disabled');
action_buttons.attr('href', "javascript:void(0)")
}
}, 500);
});
});
$(document).on('ready', function() {
var document_id = $('#download-all').data('document-id');
$(document).on("ready", function () {
var document_id = $("#download-all").data("document-id");
var intervalId; // Declare intervalId variable in outer scope
// Function to check if download is ready
function checkDownloadStatus() {
if (document_id) {
$.ajax({
url: "/download_all/button_ready/" + document_id,
type: 'GET',
dataType: 'json',
success: function(response) {
if (response.is_ready) {
$('#download-all').attr('disabled', false);
clearInterval(intervalId); // Clear the interval
}
},
error: function(request, error) {
console.error("Request error:", error);
}
});
}
if (document_id) {
$.ajax({
url: "/download_all/button_ready/" + document_id,
type: "GET",
dataType: "json",
success: function (response) {
if (response.is_ready) {
$("#download-all").attr("disabled", false);
clearInterval(intervalId); // Clear the interval
}
},
error: function (request, error) {
console.error("Request error:", error);
},
});
}
}
$(document).ready(function () {
$("#with_files_submit").click(function () {
// Check if overlay already exists; if not, create it
if ($("#overlay").length === 0) {
// Create overlay element
$("body").append(`
<div id="overlay" class="overlay" style="display: flex;">
</div>
`);
// Add CSS styles dynamically
$("<style>")
.prop("type", "text/css")
.html(
`
#overlay {
position: fixed;
top: 0;
left: 0;
width: 100%;
height: 100%;
background-color: rgba(0, 0, 0, 0.1);
justify-content: center;
align-items: center;
z-index: 1000;
display: flex;
pointer-events: none;
}
`
)
.appendTo("head");
}
// Show the overlay
$("#overlay").fadeIn(); // Show the overlay when the button is clicked
// Hide the overlay when clicking on it
$("#overlay").click(function () {
$(this).fadeOut(function () {
$(this).remove(); // Remove the overlay from the DOM after fading out
});
});
});
});
// Initial check on document ready
checkDownloadStatus();
......
......@@ -150,6 +150,28 @@ form .field-wrapper label[required="required"]::after {
min-width: 110px;
}
}
.show-actions{
display: flex;
.download-all {
display: flex;
padding: 16px;
margin: 0px;
#download-all {
margin: 7px 7px 7px 0px;
}
.download-info {
font-size: 25px;
margin: 9px 0px;
}
}
.citations .citation-modal-btn {
margin: 7px;
}
}
}
.colection-sidebar,
......@@ -192,8 +214,7 @@ form .field-wrapper label[required="required"]::after {
.work_description {
text-align: justify;
margin-top: 10px;
margin-bottom: 10px;
margin: 20px 0px 20px 0px;
}
.collection div {
......@@ -328,3 +349,31 @@ form .field-wrapper label[required="required"]::after {
.subject-parents, .session-parents, .modality-parents {
margin-top: 20px;
}
#savewidget{
position: sticky;
z-index: 1000;
top: 100px;
}
.progress-bar {
width: 100%
}
.save-warning {
color: red;
}
.agreement-wrapper div.dialog {
width: 95%;
max-width: 43em;
margin: 4em auto 0;
padding: 20px;
}
#work-entries {
.work-tab-actions {
display: flex;
gap: 10px;
}
}
\ No newline at end of file
class AgreementsController < ApplicationController
skip_before_action :check_agreement_acceptance
def deposite_agreement
return redirect_to request.referer || root_path unless current_user
end
end
......@@ -10,9 +10,7 @@ class ApplicationController < ActionController::Base
include Hyrax::Controller
include Hyrax::ThemedLayoutController
with_themed_layout '1_column'
# ToDo: Disabling terms of use feature until we can reproduce the site wide errors experienced at RUB
# before_action :check_agreement_acceptance, if: -> { ['sessions', 'homepage'].exclude?(controller_name) }
before_action :check_agreement_acceptance
private
......@@ -26,8 +24,8 @@ class ApplicationController < ActionController::Base
end
def check_agreement_acceptance
if current_user.present? && !current_user.agreement_accepted?
redirect_to root_path
if current_user.present? && !current_user.agreement_accepted?
redirect_to main_app.deposite_agreement_path
end
end
end
......@@ -10,11 +10,12 @@ module Bulkrax
include Bulkrax::DownloadBehavior
include Bulkrax::API
include Bulkrax::ValidationHelper
include HyraxHelper
protect_from_forgery unless: -> { api_request? }
before_action :token_authenticate!, if: -> { api_request? }, only: [:create, :update, :delete]
before_action :authenticate_user!, unless: -> { api_request? }
before_action :set_importer, only: [:show, :edit, :update, :destroy]
before_action :set_importer, only: [:show, :edit, :update, :destroy, :verify_and_generate_work]
with_themed_layout 'dashboard'
# GET /importers
......@@ -212,6 +213,23 @@ module Bulkrax
send_content
end
def verify_and_generate_work
entry = Bulkrax::Entry.find_by(id: params[:entry_id])
varification_status, date, report_path = latest_varification_report_details(entry)
if params[:verify]
experiment_status, report_path = VerifyCRCDatasetImport.verify_experiment_and_report(@importer.id, params[:entry_id])
end
return head :not_found unless File.exists?(report_path)
respond_to do |format|
format.html {redirect_to bulkrax.importer_path(@importer.id) }
format.csv { send_file report_path, type: 'application/csv', disposition: 'attachment' }
format.any { head :unsupported_media_type }
end
end
private
def files_for_import(file, cloud_files)
......@@ -229,7 +247,7 @@ module Bulkrax
# Use callbacks to share common setup or constraints between actions.
def set_importer
@importer = Importer.find(params[:id])
@importer = Importer.find(params[:id] || params[:importer_id])
end
def importable_params
......
......@@ -3,10 +3,17 @@
module Bulkrax
class RerunImportersController < ApplicationController
include BulkraxHelper
before_action :set_importer, only: [:rerun_failed_entries, :marked_as_complete]
before_action :set_importer, only: [:rerun_failed_entries, :rerun_panding_entries, :marked_as_complete]
def rerun_failed_entries
rerun_failed_entries_in_last_run
total_rerun_entries = rerun_failed_or_pending_entries_in_last_run("Failed")
flash[:notice] = "#{total_rerun_entries} failed jobs queued"
redirect_to importers_path
end
def rerun_panding_entries
total_rerun_entries = rerun_failed_or_pending_entries_in_last_run("Pending")
flash[:notice] = "#{total_rerun_entries} pending jobs queued"
redirect_to importers_path
end
......
......@@ -24,6 +24,8 @@ class DownloadAllController < Hyrax::DownloadsController
def build_download_file
return redirect_to format: :zip if has_zip_file?
return redirect_to format: :sh if has_shell_file?
# Created id.txt file that store timestamp when we start download
create_or_modify_ts_file
#ToDo: Can we check for this faster than getting list from S3?
list_of_objects, _total_size, format = objects_size_and_file_format
if format == 'zip'
......
......@@ -4,11 +4,11 @@ module Hyrax
module CrcDatasets
class ComplexModalitiesController < ApplicationController
include ComplexHelper
before_action :set_crc_dataset, except: %i[validate_modality_title]
before_action :set_crc_dataset
before_action :set_complex_session, only: %i[new create validate_modality_title]
before_action :set_complex_modality, only: %i[show edit update]
before_action :check_tombstone, except: %i[validate_modality_title]
before_action :authorize, only: [:new, :edit]
before_action :authorize
with_themed_layout :decide_layout
def new
......@@ -108,10 +108,6 @@ module Hyrax
redirect_to main_app.polymorphic_path(@crc_dataset)
end
end
def authorize
redirect_to main_app.polymorphic_path(@crc_dataset) unless current_user.can?(:edit, @crc_dataset)
end
end
end
end
......@@ -5,11 +5,11 @@ module Hyrax
class ComplexSessionsController < ApplicationController
include HyraxHelper
include ComplexHelper
before_action :set_crc_dataset, except: %i[validate_session_title]
before_action :set_crc_dataset
before_action :set_complex_subject, only: %i[new create validate_session_title]
before_action :set_complex_session, only: %i[show edit update]
before_action :check_tombstone, except: %i[validate_session_title]
before_action :authorize, only: [:new, :edit]
before_action :authorize
with_themed_layout :decide_layout
def new
......@@ -110,10 +110,6 @@ module Hyrax
redirect_to main_app.polymorphic_path(@crc_dataset)
end
end
def authorize
redirect_to main_app.polymorphic_path(@crc_dataset) unless current_user.can?(:edit, @crc_dataset)
end
end
end
end
......@@ -8,7 +8,7 @@ module Hyrax
before_action :set_crc_dataset
before_action :set_complex_subject, only: %i[show edit update]
before_action :check_tombstone, except: %i[validate_subject_title]
before_action :authorize, only: [:new, :edit]
before_action :authorize
with_themed_layout :decide_layout
def new
......@@ -101,10 +101,6 @@ module Hyrax
redirect_to main_app.polymorphic_path(@crc_dataset)
end
end
def authorize
redirect_to main_app.polymorphic_path(@crc_dataset) unless current_user.can?(:edit, @crc_dataset)
end
end
end
end
class UsersController < Hyrax::UsersController
# ToDo: Disabling terms of use feature until we can reproduce the site wide errors experienced at RUB
# skip_before_action :check_agreement_acceptance, only: [:accept_agreement, :discard_agreement]
skip_before_action :check_agreement_acceptance, only: [:accept_agreement, :discard_agreement]
def show
authenticate_user!
raise CanCan::AccessDenied unless (current_user.admin? or @user.id == current_user.id)
......
......@@ -30,8 +30,6 @@ module Hyrax
:keyword,
:complex_relation,
:license,
:format,
:files_size
]
self.required_fields = [
......
module BulkraxHelper
def rerun_failed_entries_in_last_run
def rerun_failed_or_pending_entries_in_last_run(status="Failed")
total_rerun_entries = 0
current_run = @importer.last_run
if current_run&.failed_records&.positive?
::Bulkrax::ScheduleRelationshipsJob.set(wait: 5.minutes).perform_later(importer_id: @importer.id)
@importer.status_info('Pending')
if current_run.failed_records.positive? || current_run.enqueued_records.positive?
total_rerun_entries = rerun_failed_or_pending_entries(@importer, status)
elsif current_run && @importer.parser.total.positive? && current_run.processed_records.zero?
::Bulkrax::ImporterJob.send(
@importer.parser.perform_method,
@importer.id
)
end
total_rerun_entries
end
@importer.entries.each do |entry|
next if entry.status == 'Complete'
def rerun_failed_or_pending_entries(importer, status="Failed")
current_run = importer.last_run
total_rerun_entries = 0
::Bulkrax::ScheduleRelationshipsJob.set(wait: 5.minutes).perform_later(importer_id: importer.id)
importer.status_info('Pending')
type = if entry.raw_metadata['model'] == 'CrcDataset'
'Work'
else
entry.raw_metadata['model']
end
importer.entries.each do |entry|
next unless entry.status == status
entry.status_info('Pending')
type = if entry.raw_metadata['model'] == 'CrcDataset'
'Work'
else
entry.raw_metadata['model']
end
entry.status_info('Pending')
if current_run.failed_records.positive?
current_run.decrement!(:failed_records)
current_run.increment!(:enqueued_records)
if type == 'Collection'
current_run.decrement!(:failed_collections)
elsif type == 'Work'
current_run.decrement!(:failed_works)
elsif type == 'FileSet'
current_run.decrement!(:failed_file_sets)
elsif type == 'ComplexSubject'
current_run.decrement!(:failed_complex_subjects)
elsif type == 'ComplexSession'
current_run.decrement!(:failed_complex_sessions)
elsif type == 'ComplexModality'
current_run.decrement!(:failed_complex_modalities)
end
"Bulkrax::CrcDataset::Import#{type}Job".constantize.send(
entry.parser.perform_method,
entry.id,
@importer.importer_runs.last.id
)
end
current_run.save
current_run.increment!(:enqueued_records)
elsif current_run && @importer.parser.total.positive? && current_run.processed_records.zero?
::Bulkrax::ImporterJob.send(
@importer.parser.perform_method,
@importer.id
if type == 'Collection' && current_run.failed_collections.positive?
current_run.decrement!(:failed_collections)
elsif type == 'Work' && current_run.failed_works.positive?
current_run.decrement!(:failed_works)
elsif type == 'FileSet' && current_run.failed_file_sets.positive?
current_run.decrement!(:failed_file_sets)
elsif type == 'ComplexSubject' && current_run.failed_complex_subjects.positive?
current_run.decrement!(:failed_complex_subjects)
elsif type == 'ComplexSession' && current_run.failed_complex_sessions.positive?
current_run.decrement!(:failed_complex_sessions)
elsif type == 'ComplexModality' && current_run.failed_complex_modalities.positive?
current_run.decrement!(:failed_complex_modalities)
end
"Bulkrax::CrcDataset::Import#{type}Job".constantize.send(
entry.parser.perform_method,
entry.id,
importer.importer_runs.last.id
)
current_run.save
total_rerun_entries +=1
end
total_rerun_entries
end
end
\ No newline at end of file
module ComplexFilterHelper
def fetch_crc_filter_params(params)
facet_filters = params[:f].present? ? params[:f].slice(*::CrcDataset::CRC_FILTER_FACET_FIELDS).permit!.to_h : {}
range_filters = (params[:range].present? && params[:range][:year_recorded_sim].present?) ? params[:range].slice(:year_recorded_sim).permit!.to_h : {}
search_params = facet_filters.merge(range_filters)
year_range_filters = (params[:range].present? && params[:range][:year_recorded_sim].present?) ? params[:range].slice(:year_recorded_sim).permit!.to_h : {}
age_range_filters = (params[:range].present? && params[:range][:complex_subject_age_itsim].present?) ? params[:range].slice(:complex_subject_age_itsim).permit!.to_h : {}
search_params = facet_filters.merge(year_range_filters).merge(age_range_filters)
end
def subject_field_mappings
......@@ -46,7 +47,7 @@ module ComplexFilterHelper
end
end
def filtered_subjects(subject_query, session_query, modality_query)
def filtered_subjects(subject_query = nil, session_query = nil , modality_query = nil)
complex_subjects = ComplexSubject.includes(complex_sessions: :complex_modalities)
if modality_query.present?
......@@ -64,7 +65,7 @@ module ComplexFilterHelper
complex_subjects
end
def filtered_sessions(subject_query, session_query, modality_query)
def filtered_sessions(subject_query = nil, session_query = nil , modality_query = nil)
complex_sessions = ComplexSession.includes(:complex_subject, :complex_modalities)
if subject_query.present?
......@@ -82,7 +83,7 @@ module ComplexFilterHelper
complex_sessions
end
def filtered_modalities(subject_query, session_query, modality_query)
def filtered_modalities(subject_query = nil, session_query = nil , modality_query = nil)
complex_modalities = ComplexModality.includes(complex_session: :complex_subject)
if subject_query.present?
......@@ -110,6 +111,13 @@ module ComplexFilterHelper
end_date = Date.parse("#{year_range['end']}-12-31")
query["session_date_recorded"] = begin_date..end_date
end
elsif key == "complex_subject_age_itsim"
age_range = search_params[key]
if age_range.is_a?(Hash) && age_range.key?("begin") && age_range.key?("end")
query["subject_age"] = age_range['begin']..age_range['end']
else
query[value] = search_params[key]
end
else
query[value] = search_params[key]
end
......@@ -126,7 +134,8 @@ module ComplexFilterHelper
facet_filters = search_params[:f] || {}
range_filters = (search_params[:range].present? && search_params[:range][:year_recorded_sim].present?) ? search_params[:range].slice(:year_recorded_sim) : {}
search_params = facet_filters.merge(range_filters)
age_range_filters = (params[:range].present? && params[:range][:complex_subject_age_itsim].present?) ? params[:range].slice(:complex_subject_age_itsim).permit!.to_h : {}
search_params = facet_filters.merge(range_filters).merge(age_range_filters)
return has_any_search_params?(field_mappings, search_params)
end
......
......@@ -30,4 +30,15 @@ module ComplexHelper
title.sub!(/-s3alias\z/i, '')
title
end
private
def authorize
case action_name
when 'show'
authorize! :show, @crc_dataset
else
authorize! :edit, @crc_dataset
end
end
end
\ No newline at end of file
......@@ -24,7 +24,7 @@ module DownloadHelper
def has_zip_file?
if work.date_modified.present?
File.exists?(zip_file_path) and local_file_last_modified > work.date_modified
File.exists?(zip_file_path) and File.exists?(ts_file_path) and ts_file_last_modified > work.date_modified
else
File.exists?(zip_file_path)
end
......@@ -42,7 +42,7 @@ module DownloadHelper
def shell_file_expired?(file_path)
if work.date_modified.present?
File.ctime(file_path).to_datetime <= (DateTime.now - 1.day) or
local_file_last_modified <= work.date_modified
(File.exists?(ts_file_path) and ts_file_last_modified <= work.date_modified)
else
File.ctime(file_path).to_datetime <= (DateTime.now - 1.day)
end
......@@ -60,6 +60,21 @@ module DownloadHelper
File.join(ENV.fetch('DOWNLOAD_PATH', "/shared/downloads"), "#{work.id}.sh")
end
def ts_file_path
File.join(ENV.fetch('DOWNLOAD_PATH', "/shared/downloads"), "#{work.id}.txt")
end
def create_or_modify_ts_file
File.open(ts_file_path, "w") do |file|
file.puts(DateTime.now)
end
end
def ts_file_last_modified
timestamp = File.read(ts_file_path).strip
DateTime.parse(timestamp)
end
def cleanup_path(file_path)
FileUtils.rm_rf(file_path)
end
......
# frozen_string_literal: true
module Hyrax
module WorkflowsHelper
# Does a workflow restriction exist for the given :object and
# given :ability?
#
# @note If the object responds to a :workflow_restriction?, we'll
# use that answer.
#
# This method doesn't answer what kind of restriction is in place
# (that requires a far more nuanced permissioning system than
# Hyrax presently has). Instead, it answers is there one in
# place. From that answer, you may opt out of rendering a region
# on a view (e.g. don't show links to the edit page).
#
# @param object [Object]
# @param ability [Ability]
#
# @return [false] when there are no applicable workflow restrictions
#
# @return [true] when there is an applicable workflow restriction,
# and you likely want to not render something.
#
# @note This is Jeremy, I encourage you to look at the views that
# call this method to understand the conceptual space this
# method covers.
#
# @todo As I noodle on this, I'm fairly certain we should be
# registering a CanCan ability check. I believe in
# promoting this to a helper method it will be easier to
# incorporate this into an ability.
#
# @see Hyrax::FileSetsController for non-view usage.
def workflow_restriction?(object, ability: current_ability)
return false if object.nil? # Yup, we may get nil, and there's no restriction on nil
return object.workflow_restriction? if object.respond_to?(:workflow_restriction?)
return false if ability.can?(:edit, object) || ability.can?(:read, object)
return object.suppressed? if object.respond_to?(:suppressed?)
false
end
end
end
\ No newline at end of file