@Terminada Why not create a deb file? How to Create DEB Packages for Debian/Ubuntu
I have cardano-node-debian. But the idea of using a deb package seems to have gotten exactly zero traction. Sadly, it seems that the community prefers to follow guides that encourage the use of massive un-auditable custom scripts rather than use their package manager for this task. The community also seems to largely prefer using custom environment files to store custom paths, setup for only a specific user. Novice users sometimes end up with multiple copies of executables and libraries in different locations on their machines and often are uncertain which ones they are actually running, especially if they follow several different guides.
You will continue to see forum questions like this:
- how to remove old cardano-cli and copy new cardano-cli?
- /usr/local/bin/cardano-node: symbol lookup error: /usr/local/bin/cardano-node: undefined symbol: crypto_vrf_ietfdraft03_secretkeybytes?
- ERROR: gLiveView failed to load common env file, Please verify set values in âUser Variablesâ section in env file?
- I assume that the cnode.sh file contains an path variable pointing to the wrong binary but shouldnât those paths be in the env file, or is it located in the bashrc file?
- cardano-cli: error while loading shared libraries: libsecp256k1?
- ERROR: Failed to load common env file?
- I found the solution, the env variable CNODEBIN was changed to the incorrect path after updating the gLiveView.
Those were just a few I could find quickly.
1 Like
On the one hand, yes!
On the other hand, we only see the operators having problems, the ones who perhaps should better not run a pool, here. Most are probably doing just fine, knowing their way around Linux (as you should if you operate servers) and can manage irrespective which manual or package manager assisted way they prefer.
1 Like
Yes and these experienced operators also donât need, or even read, a guide. My issue is with the guides whose purpose is to instruct and educate the newbies.
These guides tell users to download and run some massive custom script that pulls in a number of other custom massive scripts which collectively proceed to set / use a number of custom environment variables and install stuff in custom locations all chosen based on the whims of the unknown script author.
Anyone running a linux system can, of course, install and maintain their system however they like. But, the guides are aimed at newbies and really should follow some sort of best practice guidelines. In the end, these people end up running portions of the Cardano network which we want to be reliable and secure.
Surely you donât agree it is good to advise newbies to run these massive un-auditable scripts with all their system mutations whilst simultaneously espousing the mantra: âDonât trust, verifyâ?
So a deb package already exist (or you can build one yourself). This is the best way to go for Debian/Ubuntu users.
You can still do this even after installing the official deb package. Just specify in your run configuration file, create symlinks, etcâŚ
Guides are always easily outdated. With a package manager. you can track dependencies, manage updates, clean after uninstall, etcâŚ
Also they are only creating complications for themselves. Best to keep it simple as well.
1 Like
I agree with everything you said in your post save for the âofficialâ word. There is no âofficialâ deb package which is why my repository tells people one way to build their own.
I donât know how other people cope without offloading software upgrades to their package manager. Imagine if every software package on your system was managed like cardano-node with a different custom script to upgrade when a new version is released or a security fix rolled out.
I just checked: There are 328 individual deb packages on my very basic relay. Granted that many of these are ancillary library packages, but I can upgrade them all with âapt update && apt dist-upgradeâ. Imagine if there was 328 different scripts for me to run in order to upgrade each individual software component on my system.
I was actually hopeful there could have been some sort of debate about best practices for building packages and best practices for storing the files within the filesystem hierarchy not about whether we should use our package manager at all. This problem was solved over 20 years ago with the invention of package managers.
Probably because an âofficialâ tag implies responsibility and pressure on the maintainer. It doesnât need to be âofficialâ, only that it works for everyone, with a channel to report bugs.
Have a good read: https://www.debian.org/doc/manuals/packaging-tutorial/packaging-tutorial.pdf
Always build on a chroot as the tutorial says.
1 Like
One problem with being official is that it is required to use the stable (âbullseyeâ) compiler to make a package for the âstableâ distribution. That isnât going to work for compiling cardano-node since IOG have specified a particular version of ghc to use and this might change over time. The current stable ghc is 8.8.4. I didnât think it a good idea to embark on also packaging ghc just so that I can install the 8.10.7 version âofficiallyâ. I also couldnât find ghc-8.10.7 backported to stable or made into a deb for any other debian version.
Hence I compile cardano-node a very non-official way:
debuild --prepend-path "$HOME/.cabal/bin:$HOME/.ghcup/bin" -us -uc;
which manually overrides the debuild path so that it can use the build userâs locally installed ghc.
I am not knowledgeable enough to know the best way to do such things which is why I really had hoped that someone else would build deb packages and show me how to do it properly.
Arch in my system runs ghc 9.0.2 so while Debian requires an upgraded version of ghc Arch requires a downgraded version. You can perhaps create a deb file for âBullseyeâ. The source file for ghc 8.10.7 exist: GHC 8.10.7 download â The Glasgow Haskell Compiler
Just port it to Debian 11. Then you can have an âofficialâ package. But as I can see it you installed version 8.10.7 with either ghcup or cabal. That is also an âofficialâ way of doing it.
I think this is completely sensible. You are just adding the ghc executables to your own user environment path which prevents other users to use it.
The way I think it, if you can install the software without a problem and it works, then you can write a system package for by just writing in a make file way you installed it. If at some point you found a better way of doing things, then just write an updated version of the package. That is why you have git and versioning systems.
1 Like
@Terminada Iâve been reading the deb packaging tutorial. You can in fact declkare the prepend-path command in /debian/rules
Also itâs probably best to package ghc 8.10.7 as well so that you can just add it to a Build-Depends policy when writing as node package.
I suggest you open a new, separate, discussion thread regarding packaging the node into deb because weâve pretty much hijacked this thread with our deb discussion. I would do it but Iâm mainly on Archlinux. Also the title of this thread might just be ignored by a deb developer. I am considering installing Ubuntu in a virtual machine and having a go to create a cardano-node deb package myself. A little knowledge of dpkg wonât hurt. If you have a thread regarding this we can further discuss it there.
1 Like
I canât remember the details now, but I recall there being some reason why I couldnât get that method to work. I do recall trying that.
Yeah, I know but it seemed like extra headache so I gave in and just manually overrode the compiler with the prepend-path option. Also, I had the impression that IOG was likely to change the ghc version compatibility over time, so I thought it might become a moving target.
Agree.
I am a bit focused on writing some basic Haskell code for contributory evidence for a CIP I am trying to author. I havenât spent much time on trying to sort out the proper way to build the deb package because it seemed like nobody was interested in using their package manager. You are about the only person who has paid it any attention and you use Arch rather than Debian / Ubuntu.
I will endeavour to have a go at packaging ghc 10.8.7 into a deb whilst continuing to hope that IOG moves to using 9.2.x soon so that the haskell nonmoving-gc works properly. Iâll start a new thread when I can make some more progress. But I am not a shadowy super-coder and I donât even work in IT so my skillset is not as good as I wish.
Thanks for your interest and suggestions.
If it didnât work, my guess is this: the deb package is owned by root so when you issue --prepend-path â$HOME/.cabal/bin:$HOME/.ghcup/binâ it is looking at $HOME of root for your ghc executables which of course doesnât exit since itâs in userâs $HOME directory.
This is why I think packaging the ghc 8.10.7 first is probably the better idea. Then you can prepend the executables where the package installs it.
You probably need to package a new ghc version anyway if IOG starts using another version. I donât have information why Cardano has to use specifically a certain ghc compiler version.
I am in the same boat. Iâm purely a computer coding hobbyist. Anyway just open a thread when youâre ready. Cheers!
1 Like
@Terminada Coming back to our deb discussion, I installed Debian in a virtual machine, initially âBullseyeâ 11, but I just miss having a rolling release so I decided use the testing repository instead. Also itâs closer to what Iâm used to at Archlinux.
Iâve looked at how the package in Archlinux by looking at the PKGBUILD file to see how cardano-node installs. The package source already has static binaries which means you donât need ghc-8.10.7 since you donât need to build the binaries. It simply installs the different elements in their directories and adds configuration files for the user and system.
Iâm going to attempt to write a deb package using the same source, use the same configuration files (thanks to the maintainer there) from the Archlinux package, and write the deb rules along similar lines.
Iâve written this post to encourage you to try it as well.
I had thought about just downloading the binaries built by IOG and I think that would be the way to go for most software. That would certainly simplify the task to just a packaging job.
However, we are talking about Cardano and the âdonât trust, verifyâ motto we all espouse. I think for stake pool operators at least, it is important to compile the software yourself.
For those that just want to run a node (relay) without being a pool operator, I do have compile binary debs that others can just âapt installâ here: https://terminadapool.github.io/deb/
Actually, people can also check my compilations themselves to be sure I am not doing something nefarious because they can build the debs using my repository instructions and confirm that their debs have the exact same hash. This is even though the deb build process does do things like hardening where compiled shared c/c++ library access memory locations are randomised, the binary debs are exactly reproducible.
That is exactly how I build my debs. I then upload my built debs to my internal deb repository and then simply apt install them to each relay and block producer node. This way there is no extra build chain on the running machines. There is no confusion about which binary is running or which library it is using or any custom paths - environment variables. Simple and much easier to maintain.
I try to stick with the current stable Debian (Bullseye 11) because it is rock solid and I know the security team is actively providing security updates for every piece of software on an urgent basis.
What security improvement do you see in compiling yourself without reading and analysing the source code you are compiling in detail?
I half disagree with this. Of course a good stake pool operator will definitely need to have haskell for maintanance but I think we could attract more pool operators if you can just install a node. âdonât trust, verifyâ happens. All the binaries come from upstream, and then just calculate an md5sum for the sources so that they are in fact downloading legit binaries. In essence thatâs what dh_md5sums does and how you verify.
Yes that is a good question. I canât say that I have enough ability myself to do this anywhere close to adequately. I rely on the fact that many eyes are looking at the open source code, of which my eyes are just one set.
However, if you donât compile the code yourself then you are still placing an extra person or entity of trust into the process.
But we do that all the time!! Thatâs why we use computers, higher abstraction. Otherwise you use Gentoo which compiles every package before installing them. From ground up.
Yes, but there is a difference when comparing a niche product like Cardano that only a few thousand people are running vs every other piece of software on my Debian system that is being run by millions. Furthermore all the other packages on my system have had the Debian security team audit them as well as Ubuntu, Mint and all other Debian off-shoot distributions. That gives me a bit more confidence about every other package on my machines running Debian stable (Bullseye).