Request for Dataset of Mempools

Dear Everyone,

I am a PhD student at the Hong Kong University of Science and Technology. My research paper is focused on Cardano mempools. Unfortunately, I have limited resources and cannot run and maintain a Cardano node to store snapshots of the mempools(Pending Transactions). I need a snapshot of Mempool transactions for each block throughout a couple of days.

If anyone can provide assistance or help me make contact with relevant individuals, it would be greatly appreciated. Please feel free to contact me at your convenience so that we can discuss the terms.

Thank you for your kind attention.

Group Website: Amir Goharshady - ALPACAS Research Group
My Official Contact Information:
Soroush Farokhnia
sfarokhnia@connect.ust.hk

1 Like

Hey @Soroush_Farokhnia

That sounds really interesting :slight_smile:

Maybe there is someone from the Cardano foundation which could help you out in such a case or maybe connect you with someone.

@lodl_de @werkof

1 Like

I cannot directly help out (yet) but may point you to two directions:

First for a live and historic view of blocks look at https://eutxo.org/
There you can see TXs added to one (this projects) node mempool, then backed into blocks.

What is easily possible is to look at all TXs backed into each adopted and finalized block.
You don’t need your own node, but can just query public endpoints like Koios API Documentation
and then more details on these TXs from different other endpoints Koios API Documentation
Alternatively you can also query Block transactions | Blockfrost Development Hub

Instead if you want a list of TXs sitting and waiting into a mempool right when a block gets produced, the answer is: there is no “the one mempool”
Ideally and in non congested network phases all mempools have a very similar list of waiting TXs. But then they highly probably already have a different order, as a TX submitted at the same 50 milliseconds range from Hong Kong will reach US nodes with some delay. And another TX submitted from US around the same time will reach Asia only after the Hong-Kong one. No problem for the block production process, but not possible to provide you that one snapshot you may would like to have.

Then in congested network phases the situation becomes even more complex. Some block producers may would accept more TXs but their default mempool size limit prevents these TXs to be added. in such situations global latency results in quite different mempool indexes.

There is also https://mempool.cardanoapi.io/
an interesting project, but I need to review and get in touch with them (finally as promised for many weeks already and sorry) because some data interpretation there imo could be improved

4 Likes

Dear @werkof,

Thank you for your comprehensive and courteous response. I appreciate your insights. Thanks for introducing services to gather historical data on the chain. I agree that obtaining a local view(not a global view) of pending transactions in real-time without a running node is not feasible. Also,

However, I apologize that I did not provide enough information regarding my requirements.
My goal is not to receive a real-time list of pending transactions before a block gets mined, and I understand that increasing the size of the Mempool is not feasible. Instead, I am looking for a service or entity to store a snapshot of their Mempool(with their default settings) for a few days, which I can access later. I would be happy to help them and provide the required scripts.

Thanks.

P.S: I’ve worked with many blockchains and their software, I admit that Cardano way of having different modules(units?) is necessary due to its design and protocol specs.

1 Like

Sorry for the delayed answer.

Technically it is possible to fetch a block producers mempool contebt at any time. And then ofc store that content in some format on disk or database.

For example look at
https://ogmios-python.readthedocs.io/en/latest/examples/get_mempool_contents.html

The next question is: for how many different block producers or which absolute or relative amount of blocks you would need this?

1 Like

Hi @werkof,

Not at all, sir. Anyone who uses technical forums understands that People in technical forums answer out of generosity and interest, not duty. Thanks for your reply.

Yes, you are right. Please correct me if I’m wrong, but as I understand working with Ogmios is not possible if you don’t have access to a running node. I haven’t found any API that gives me access (i.e. koios does not support Mempool).

We need data from a block producer for a week, and it would be great to have snapshots of every block. But that’s too much computation and I/O. So, we’re thinking every 5 blocks(or close to this) would be great (or in another word taking snapshot every 5 seconds).

Thanks in advance.