Fedora Migration Paths and Tools Project Update: July 2021 - Duraspace.org


						
			

			
					Projects
	DSpace
	Fedora
	VIVO
	Who’s Using



	Services
	ArchivesDirect
	DSpaceDirect
	DuraCloud



	Community
	Our Users
	Community Programs
	Service Providers
	Strategic Partners



	Membership
	Values and Benefits
	Current Members
	Financial Contributors
	Become a Member



	Support
	Choosing a Project
	Choosing a Service
	Technical Specifications
	Wiki
	Contact Us



	News & Events
	Latest News
	Event Calendar
	Webinars
	Monthly Newsletter




			

			
					About DuraSpace
	Projects
	Services
	Community
	Membership
	Support
	News & Events

			

			
				
				 
									
						
							
	
	
	
			
				
			


	

	
	
	
			Projects
	DSpace
	Fedora
	VIVO
	Who’s Using



	Services
	ArchivesDirect
	DSpaceDirect
	DuraCloud



	Community
	Our Users
	Community Programs
	Service Providers
	Strategic Partners



	Membership
	Values and Benefits
	Current Members
	Financial Contributors
	Become a Member



	Support
	Choosing a Project
	Choosing a Service
	Technical Specifications
	Wiki
	Contact Us



	News & Events
	Latest News
	Event Calendar
	Webinars
	Monthly Newsletter




	

	
		 
			
		

	

	
	
	
			


	


						

					

				 	

	

		Home » Latest News » Fedora Migration Paths and Tools Project Update: July 2021
		
		
							
						
		
		Fedora Migration Paths and Tools Project Update: July 2021

					 Posted on July 28, 2021 by David Wilcox
	
			 
					
	
		This is the latest in a series of monthly updates on the Fedora Migration Paths and Tools project – please see the previous post for a summary of the work completed up to that point. This project has been generously funded by the IMLS.

We completed some final performance tests and optimizations for the University of Virginia pilot. Both the migration to their AWS server and the Fedora 6.0 indexing operation were much slower than anticipated, so the project team tested a number of optimizations, including:

	Adding more processing threads
	Increasing the size of the server instance 
	Using a separate and larger database server 
	Using locally attached flash storage


Fortunately, these improvements made a big difference; for example, ingest speed was increased from 6.8 resources per second to 45.6 resources per second. In general, this means that institutions with specific performance targets can use a combination of parallel processing and increased computational resources. Feedback from this pilot has been incorporated into the migration guide, updates to the migration-utils to improve performance, updates to the aws-deployer tool to provide additional options, and improvements to the migration-validator to handle errors.

The Whitman College team has begun their production migration using Islandora Workbench. Initial benchmarking has shown that running Workbench from the production server rather than locally on a laptop achieves much better performance, so this is the recommended approach. The team is working collection-by-collection using CSV files and a tracking spreadsheet to keep track of each collection as it is ingested and ready to be tested. They have also developed a Quality Control checklist to make sure everything is working as intended – we anticipate doing detailed checks on the first few collections and spot checks for subsequent collections.

As we near the end of the pilot project phase of the grant work we are focused on documentation for the migration toolkit. We plan to complete a draft of this documentation over the summer, after which this draft will be shared with the broader community for feedback. We will organize meetings in the Fall to provide opportunities for community members to provide additional feedback on the toolkit and make suggestions for improvements.

	 
						
	
				Tags: Blog, Fedora, Fedora Repository, News, Open source
	
	 
						
		
													
 				
					 

		

	
		DuraSpace Articles
	Recent Articles
	RSS Feeds



Tags
Announcements (35)
Blog (397)
Cloud (21)
COAR (7)
Communication (425)
community (6)
Conferences (86)
Data curation (91)
DSpace (209)
DSpace 7 (39)
DSpaceDirect (13)
DuraCloud (46)
DuraSpace (370)
DuraSpace digest (346)
education (5)
Events (88)
Fedora (9)
Fedora Repository (170)
Governance (5)
Higher education (40)
Hydra (62)
Islandora (75)
Linked data (106)
LYRASIS (58)
LYRASIS Digest (17)
meetings (14)
NDSA (9)
News (362)
Open access (370)
Open data (348)
Open Repositories (13)
Open source (370)
Preservation and archiving (260)
professional development (7)
Registered Service Provider (8)
Repository (247)
Samvera (14)
Scholarly publishing (296)
SPARC (12)
Technology (151)
VIVO (165)
VIVO Camp (7)
VIVO Conference (15)
VIVO updates (20)
Web seminar (33)



	


	
 

 
				
					
						
							
									About
	About DuraSpace
	History
	What We Do
	Board of Directors
	Meet the Team
	Policies
	Reports



	Community
	Our Users
	Community Programs
	Service Providers
	Strategic Partners



	Membership
	Values & Benefits
	Current Members
	Financial Contributors
	Become a Member



	Support
	Choosing a Project
	Choosing a Service
	Technical Specifications
	Wiki
	Contact Us



	News & Events
	Latest News
	Event Calendar
	Webinars
	Monthly Newsletter




							
						

					
 
					
						
							
								 This work is licensed under a Creative Commons Attribution 4.0 International License