Shore-MT: A Scalable Storage Manager for the Multicore Era

Goal

The database research community needs a full-featured storage manager which can scale to utilize today's highly parallel multicore hardware. Shore-MT has provided us this functionality and we want other researchers to benefit from it as well.

Latest News

17/12/2010: New public release of Shore-MT (Shore-MT-6.0). A compatible version of Shore-Kits will soon be available. Documentation is now in place and the systems compiles both with SparcV9 and X86_64 architectures (see download page).

Description

Shore-MT is an experiment test-bed library for use by researchers who wish to write multi-threaded software that manages persistent data. Shore-MT as a storage engine provides the following capabilities:

transactions with ACID properties, with ARIES-based logging and recovery, primitives for partial rollback, transaction chaining, and early lock release,
prepared-transaction support for two-phased commit,
persistent storage structures : B+ tree indexes, R* trees (spatial indexes), and files of untyped records,
fine-grained locking for records and B+ tree indexes with deadlock detection, optional lock escalation and optional coarse-grained locking,
in-memory buffer management with optional prefetching,
extensible statistics-gathering, option-processing, and error-handling facilities.

This software runs on Pthreads, thereby providing its client software (e.g., a database server) multi-threading capabilities and resulting scalability from modern SMP and NUMA architectures, and has been used on Linux/x86-64 and Solaris/Niagara architectures.

Who are we?

Shore-MT is developed and maintained by the members of the DIAS Laboratory at EPFL, Switzerland, as well as the StagedDB/CMP project group at Carnegie Mellon University in collaboration with the University of Wisconsin.

Source and Documentation

The source code can be downloaded from our download page. The documentation can be found online here.

Shore-MT mailing list

Please visit the webpage of shore-mt mailing list.

Publications

Below you can find a list of publications related with Shore-MT.

[VLDBJ11]
Scalability of write-ahead logging on multicore and multisocket hardware.

Ryan Johnson, Ippokratis Pandis, Radu Stoica, Manos Athanassoulis, and Anastasia Ailamaki.
In the VLDB Journal, 20(6), 2011.
PDF [1m]

[VLDB11]
PLP: Page Latch-free Shared-everythin OLTP

Ippokratis Pandis, Pinar Tozun, Ryan Johnson, and Anastasia Ailamaki.
In Proceedings of the VLDB Endowment, volume 4, 2011.
PDF [824k]
Presented at Seattle, September 2011.

[SIGMOD11]
A Data-oriented Transaction Execution Engine and Supporting Tools

Ippokratis Pandis, Pinar Tozun, Miguel Branco, Dimitris Karampinas, Danica Porobic, Ryan Johnson, and Anastasia Ailamaki.
In Proceedings of ACM SIGMOD International Conference on Management of Data, 2011.
PDF [657k]
Presented at Athens, June 2011. Won the BEST DEMO AWARD.

[VLDB10a]
Data-Oriented Transaction Execution

Ippokratis Pandis, Ryan Johnson, Nikos Hardavellas and Anastasia Ailamaki.
In Proceedings of the VLDB Endowment, volume 3, number 1, 2010.
PDF [406k]
Presented at Singapore, September 2010.

[VLDB10b]
Aether: A Scalable Approach to Logging

Ryan Johnson, Ippokratis Pandis, Radu Stoica, Manos Athanassoulis and Anastasia Ailamaki.
In Proceedings of the VLDB Endowment, volume 3, number 1, 2010.
PDF [326k]
Presented at Singapore, September 2010.

[VLDB09]
Improving OLTP Scalability using Speculative Lock Inheritance

Ryan Johnson, Ippokratis Pandis and Anastassia Ailamaki.
In Proceedings of the VLDB Endowment, volume 2, number 1, pages 479-489, 2009.
PDF [330k]
Presented at Lyon, France, August 2009.

[TR09]
Improving OLTP Concurrency through Early Lock Release

Manos Athanassoulis, Ryan Johnson, Anastasia Ailamaki and Radu Stoica.
EPFL Technical Report EPFL-REPORT-152158, March 2009.
PDF [623k]

[EDBT09]
Shore-MT: A Scalable Storage Manager for the Multicore Era

Ryan Johnson, Ippokratis Pandis, Nikos Hardavellas, Anastassia Ailamaki and Babak Falsafi.
In proceedings of the 12th EDBT, Saint Petersburg, Russia, 2009.
PDF [332k]

[DAMON08]
Critical Sections: Re-emerging Scalability Concerns for Database Storage Engines

Ryan Johnson, Ippokratis Pandis, and Anastassia Ailamaki.
In proceedings of the 4th DaMoN, Vancouver, Canada, June 2008.
PDF [251k]

[TR08]
Shore-MT: A Quest for Scalability in the Many-Core Era

Ryan Johnson, Ippokratis Pandis, Nikos Hardavellas, and Anastassia Ailamaki.
Carnegie Mellon University Technical Report CMU-CS-08-114, April 2008.

Acknowledgments

This work is partially supported by Sloan research fellowship, NSF grants CCR-0205544, IIS-0133686, and IIS-0713409, an ESF EurYI award, and SNF funds.

[VLDBJ11]	Scalability of write-ahead logging on multicore and multisocket hardware.
	Ryan Johnson, Ippokratis Pandis, Radu Stoica, Manos Athanassoulis, and Anastasia Ailamaki. In the VLDB Journal, 20(6), 2011. PDF [1m]
[VLDB11]	PLP: Page Latch-free Shared-everythin OLTP
	Ippokratis Pandis, Pinar Tozun, Ryan Johnson, and Anastasia Ailamaki. In Proceedings of the VLDB Endowment, volume 4, 2011. PDF [824k] Presented at Seattle, September 2011.
[SIGMOD11]	A Data-oriented Transaction Execution Engine and Supporting Tools
	Ippokratis Pandis, Pinar Tozun, Miguel Branco, Dimitris Karampinas, Danica Porobic, Ryan Johnson, and Anastasia Ailamaki. In Proceedings of ACM SIGMOD International Conference on Management of Data, 2011. PDF [657k] Presented at Athens, June 2011. Won the BEST DEMO AWARD.
[VLDB10a]	Data-Oriented Transaction Execution
	Ippokratis Pandis, Ryan Johnson, Nikos Hardavellas and Anastasia Ailamaki. In Proceedings of the VLDB Endowment, volume 3, number 1, 2010. PDF [406k] Presented at Singapore, September 2010.
[VLDB10b]	Aether: A Scalable Approach to Logging
	Ryan Johnson, Ippokratis Pandis, Radu Stoica, Manos Athanassoulis and Anastasia Ailamaki. In Proceedings of the VLDB Endowment, volume 3, number 1, 2010. PDF [326k] Presented at Singapore, September 2010.
[VLDB09]	Improving OLTP Scalability using Speculative Lock Inheritance
	Ryan Johnson, Ippokratis Pandis and Anastassia Ailamaki. In Proceedings of the VLDB Endowment, volume 2, number 1, pages 479-489, 2009. PDF [330k] Presented at Lyon, France, August 2009.
[TR09]	Improving OLTP Concurrency through Early Lock Release
	Manos Athanassoulis, Ryan Johnson, Anastasia Ailamaki and Radu Stoica. EPFL Technical Report EPFL-REPORT-152158, March 2009. PDF [623k]
[EDBT09]	Shore-MT: A Scalable Storage Manager for the Multicore Era
	Ryan Johnson, Ippokratis Pandis, Nikos Hardavellas, Anastassia Ailamaki and Babak Falsafi. In proceedings of the 12th EDBT, Saint Petersburg, Russia, 2009. PDF [332k]
[DAMON08]	Critical Sections: Re-emerging Scalability Concerns for Database Storage Engines
	Ryan Johnson, Ippokratis Pandis, and Anastassia Ailamaki. In proceedings of the 4th DaMoN, Vancouver, Canada, June 2008. PDF [251k]
[TR08]	Shore-MT: A Quest for Scalability in the Many-Core Era
	Ryan Johnson, Ippokratis Pandis, Nikos Hardavellas, and Anastassia Ailamaki. Carnegie Mellon University Technical Report CMU-CS-08-114, April 2008.