Seagate CORTX Challenge

News

Seagate CORTX Challenge

UCD CS Women@CompSci hosted this event on 17 November 2021 in the School of Computer Science.

The aim was to provide an opportunity for students to meet and have fun, while sharing an interesting intellectual challenge, and to foster a sense of belonging in the school and to promote diversity and inclusion. 28 students took part, from first year Undergraduates to Masters and PhD students. Our partner Seagate designed and ran the challenge and sent along three mentors to run the challenge: Patrick Hession (a UCD CS grad) and two from Seagate’s UK European R&D team, Ganesan Umanesan and David Vasiliauskas. On behalf of UCD W@CS, Dr Fatemeh Golpayegani, Assistant Professor, UCD CS welcomed everyone and thanked Seagate for their support.

Over 3 hours in small teams, students embarked on a real-world engineering challenge supporting each other and with help from Seagate mentors.

The Science of Storage

The world runs on data these days, and more and more of it is being retained and stored than ever before. Yet the hard limits imposed by the laws of physics have meant that the science of storage has had to become more creative in order to keep up. In addition to developing more exotic ways of storing bits on drives, the software that manages storage has become an important part of the solution. New storage platforms can use available storage more efficiently, be optimized for specific uses (such as running simulations), and be designed to take advantage of new developments in the hardware to ensure that data storage, recovery, and retrieval is as efficient, effective, and fast as possible.

Storage innovations and challenges

CORTX is Seagate’s mass-capacity object storage, an open-source project designed to demonstrate how to best marry innovations in storage hardware with innovations in storage software. One of these innovations is something called FDMI (file data manipulation interface). One of the challenges that data scientists face is performing data analytics on new data in a timely and efficient manner. FDMI allows data scientists to subscribe to event notifications and then perform custom manipulations or analyses on newly created (or updated) data.

The challenge

Participants were asked to create a Python script that would keep track of the 40 most popular words in a folder of their choice on their desktop in order to emulate the FDMI functionality of CORTX. FDMI is a publish subscribe mechanism by which external parties can monitor for “interesting” events without having to expensively poll. This is unique to CORTX as with other object storage systems you would have to dive deep into the code in order to add this functionality whereas with CORTX you can just plug in FDMI. So the idea for the challenge was to emulate a storage system using a local folder on your machine and then create the python script that would "listen" to your folder and update the word count as files were removed/added to this folder. 5 out of the 8 teams managed to complete the challenge within the 2-hour window we gave them. One team even managed to add an interesting printing function to make their results look nice.

Keeping everyone safe

We limited numbers to allow social distancing and gave breaks for participants to go outside (with drinks and snacks provided by Seagate). All participants had to

wear face coverings and show COVID vaccination certificates. Each participant received a Seagate T-shirt!