Holy crap! I did not see those two on Reddit. I had posted to Reddit and was monitoring those comments but didn't know of these. Thanks! Although, I'm sad both of those threads got more comments then the original one I posted. Not that it really matters in life. Heh.
And yes, I saw that pull request but I didn't know what HN was at that time. After I saw that comment, and before they said HN = Hacker News, I started Googling around to see if I could find where my repo was being mentioned. That is how I found this post in HN.
This guide contains (at least) inaccurate statements. It's oversimplifying and omitting important things while putting emphasis on some exotic details where defaults would be sane enough.
"One key, the public key, can only encrypt data, not decrypt it" - this is cryptographically inaccurate. One should use it that way, though.
"Identity is verified by encrypting and decrypting data that both the client and server know". That's not how signing works. The crucial part of the process is to first establish an encrypted channel, then choose something random (not used previously) and finally verify the signature is correct. What exactly would be the data that "both the client and server know"?
"If you're sure there is nobody listening between the client you're on and your server, you can use ssh-copy-id to transfer and append the public key". Almost never happens unless you're in the server room, connected point-to-point. But then the remark about someone listening doesn't make sense. And not a single word about verifying server's fingerprint in the whole guide.
"Keep in mind doing this means you can't use the key for automation because you'll have no way to send the passphrase in your scripts". Unless you use an ssh-agent.
"umask is a Bash built-in which means a user can change their own umask setting". Wrong implication here. umask is a syscall. It doesn't matter what you use to call it. And what if you're using a different shell?
Suggesting painful default umask instead of just doing `chmod go-rwx $HOME` and adjusting /etc/adduser.conf is debatable.
Making this guide "distribution agnostic" is IMHO futile and there already are some distribution-specific guides like https://debian-handbook.info/.
Upgrade the version of the browser to an approved version by obtaining software from the vendor or other trusted source. Method 1: View the following registry key: HKLM\Software\Mozilla\Mozilla Firefox\CurrentVersion Method 2: Search for the firefox.exe file using the search feature of the operating system. Examine the files properties for the product version (not the file version. For Windows OS, determine the version of the file by examining navigating to Properties/Version/Product Version. Examine for all instances of firefox.exe that are present on the endpoint. Criteria: If the version number of the firefox.exe file is less than 50.1.x (or ESR 45.7.x), this is a finding.
How about clicking on "About Firefox" in the menu? "Other trusted source" - like softonic, yes?
As someone who has had to implement STIG, that is not a realistic method. This stuff is implemented via automation over hundreds of nodes at a time, and having to click through anything in automation is slow and error prone.
STIG itself is just a fancy check-list. Ideally, real world implementation is automated via something like OpenSCAP.
> “One key, the public key, can only encrypt data, not decrypt it" - this is cryptographically inaccurate. One should use it that way, though.
No, one shouldn’t. To be even more pedantic, verifying a signature under PKI is, in fact, “decrypting with the public key” something (a hash, usually) that was encrypted with the private key. When considering the complete set of PKI operations (encryption + signing), both the public and private parts of a key pair are used in both their enciphering and deciphering capacities.
"No, one shouldn’t" suggests that you don't use public key when encrypting data. "Verifying a signature […] is […] “decrypting with the public key” is the part I meant when writing that the original statement is "cryptographically inaccurate" - it's algorithmically feasible. But also mind that this specific kind of encryption has a specific name - "signing". You can't state that: "no, one shouldn't use public key for encrypting data".
> Now you need to append the contents of the public key ~/.ssh/id_ed25519.pub to the ~/.ssh/authorized_keys file on the target server. You'll want to do this in a secure way since the public key gives access to your server.
The fact you have the private key permits access, not possession of the public key even though the public key is the one you add to the file on the server side (asymmetric cryptography). The "secure way" may also suggest that you have to keep your public key secret which is not true.
I believe the statement is most charitably interpreted as the channel must be secure against modifications so that when you are attempting to put the public key in the file that some other contents aren't inserted instead.
On the other hand, SSH gives you just that if you verify the fingerprint which is not mentioned in the guide. MITM-proof channel is needed if one gains any kind of shell access because then public key substitution can happen in various ways, even without the user him/herself explicitly editing the file. Furthermore, if one allows MITM when accessing shell, substituting the public key is just one of very serious security problems.
Certainly many other operations require a tamper-proof channel, and many more things can go wrong without one but I believe the author was trying to indicate the specific requirement here rather than making a specifically incorrect statement.
I simplified things that would be too heavy to get into. The guide is not intended to teach everything about security -- that would be too much. The goal is to cover enough to give the reader a high level understanding. Once they have a basic understanding they can research more if they desire.
Can you tell me some of the "exotic details where defaults would be sane enough"? I can amend the guide.
Are you saying a public key can be used to decrypt data it encrypted? Or are you saying a public key could also be used to decrypt data that the private key encrypted?
You mention "server room". This guide is intended for a person running a simple server in their home. Hopefully SAs securing a large scale environment are not using information from GitHub. :/
I made some other updates to the guides that I hope address your other concerns?
Regarding your distribution agnostic comment, I do not see value in distribution specific guides on hardening. Sans a few edge distributions, most distributions are similar enough that the hardening steps are the same. It is okay to look for distribution specific documentation on how to install the distribution but it hurts the cause having distribution specific hardening guides.
Also, I want more folks to use Linux. Most distributions are so similar there is no value in having so many distribution specific guides -- all it does is create unnecessary confusion and steer potential prospective users away from Linux.
> The guide is not intended to teach everything about security -- that would be too much.
On the other hand, the objectives list contains: "this guide will attempt to cover as many of them as possible". Despite that, it still misses basic security rules.
"A desktop class computer [...] That I want to be able to SSH to remotely from unknown computers and unknown locations (i.e. a friend's house)."
SSH from an unknown computer? Also see my original comment about what is missing in description of a situation when connecting to your server from a new system.
> Are you saying a public key can be used to decrypt data it encrypted? Or are you saying a public key could also be used to decrypt data that the private key encrypted?
See my discussion with derefr in this thread.
> You mention "server room". This guide is intended for a person running a simple server in their home.
The "server room" here means you are sitting next to your server and can connect point-to-point. It doesn't mean you need a backup diesel generator.
And if it is really about a home server, concerns like "make sure nobody is listening" don't make sense.
> I made some other updates to the guides that I hope address your other concerns?
I've seen you accepted pull requests of some contributors that fixed the basics for you. Because they were generated right after discussion in this thread they concern things discussed here.
> Also, I want more folks to use Linux. Most distributions are so similar there is no value in having so many distribution specific guides -- all it does is create unnecessary confusion and steer potential prospective users away from Linux.
How many (which) distros have you managed as a professional sysadmin to state that with high confidence?
I'm quite sure that a beginner will just choose Ubuntu and proceed to something like https://linuxjourney.com/. If someone becomes more than a hobbyist one will hopefully go and read something from a trusted source, written by sysadmins with a good grasp of contemporary cryptography involved.
The goal is to cover as many things as possible, not everything. Security is a very deep and complex topic -- no single document can cover everything. I want to wet the readers appetite to get them interested enough to learn more.
Thanks for that link -- never seen it before.
I was an SA for a Fortune 10 company for 5+ years supporting 150k+ servers but we don't need to get into comparing resumes here. I appreciate your time and don't want to waste more of it. However, if would like to continue the discussion maybe we can do it on GitHub? I don't care for HNs commenting style. Thanks!
For umask, it actually does matter that it's a built-in. If it was an external binary instead, it would not be able to set the umask of the calling process (shell) which is what is important.
I see no point, there are enough guides already and I don't consider myself competent enough in that field to teach others. I think it shouldn't be about fixing every particular mistake in the guide. It's about changing attitude to "I won't write something unless I have a lot of experience". Otherwise it becomes a cat and mouse chase and managing basics for the author.
Yes! This guide is good for securing maybe a personal server, but any business systems should use a server hardening standard that has industry mindshare (CIS, STIG, etc.) I can speak from personal experience that QSAs give you a very skeptical look when you say "our security standard is homebrewed."
Sure but the point still stands. Why should anyone trust that you, or your company, is any good at security compared to say CIS?
If you have homegrown security and can show your QSA your detailed policy document and that it's a superset of CIS, STIG, NIST, etc. with documented exceptions then it'll be no problem.
I avoid homegrown whenever possible because it's a rabbit hole that never ends. If you instead say CIS level 2 then you can clearly define when you've done enough.
Because, checked boxes don’t mean much. CIS level 2 = a lot of checked boxes. Using cyber security frameworks is great, but some of the most compliant and “advanced” organizations have the worst legacy cruft you can imagine. We work with orgs all the time and organizations that use these frameworks with expert guidance can easily secure their most critical assets while only implementing the right parts of a framework. And the best frameworks have risk based targeting for maturity levels (NIST Cybersecurity) of various activities. These frameworks can end up being a bit of security theatre if you are just implementing it for the sake of “having security”. Guess I am just jaded after breaking software and networks for over a decade. Some of the most secure organizations have very adaptive security practices that focus on application security. Some of the worst are ISO, CIS, STIG policy template hardened blah blah blah. Just don’t put these frameworks and policies on a pedastal. The real security work happens in the margins.
Their hardened images are fine but there are two issues. First, you'll need to test / dev with them as full implementations of CIS can break things (though not as bad as the STIGs) and, IIRC, they charge extra for you to use their images on public clouds.
I am in the process of going through one of their benchmarks right now. They have some good stuff but I'm on page 130 of 431 and so far I haven't come across anything that needed to be changed from a standard Debian install.
No mention of shipping off logs to another place? It's probably good to assume someone will gain access and make after-the-fact forensics a primary concern as well.
A lot of hardening guides skip the long tail for security.
Which is to say: So you've shipped logs off, so then what? How are you going to monitor those regularly, what are you looking for, how are you going to make sure important information stands out?
Many people set up remote logging and then never check the logs until after there is an issue. An unread log isn't useful. Logs that are too spammy aren't going to be read.
Nothing about SELinux or AppArmor? While these are enabled by default on some of the big distros I think a basic guide on managing and troubleshooting would be beneficial.
Yeah, the best impact would come from using SELinux, AppArmor, and GRSec. Everything else is just tweaking defaults, which for up-to-date software probably isn't going to impact your security much at all.
I don't disagree but you gotta start with the basics. SELinux can be rather advanced/complex and probably warrants its own guide. I still have to learn it better before I try to write a guide on it.
You don't have to use a closed source TOTP client such as Google's. There's RedHat's FreeOTP and there's another open source one called antOTP. You can find them on F-Droid as well.
There have been TOTP PAM modules for ages (these work on a Linux client and Linux server via e.g. SSH). You can even add YubiKey to PAM. Same for BSD Auth and macOS.
Try the "scan code" button which should activate the camera and the app will crash. I just tested it with my iPhone and verified. The iOS App Store application was last updated 4 years ago. There's also comments there saying it doesn't work anymore.
Reading / using existing codes might work properly, or manually inputting them.
Interestingly my wife's XS is having the problem you describe but not my X.
I've sent a note to the maintainers, but there's an open git issue for it. Not sure when their commits will roll to the store.
I'm told by multiple colleagues at redhat that they've raised this with internal support, so it should be sorted eventually. Lots of iphones in redhat and this is the app they officially support for TFA internally.
When I was researching, literally every article I read mentioned Google'S TOTP PAM module. I can't even find any other ones that have active development. Do you know of any? Would love to use another one if it works with my iOS authenticator app.
If you have an NFC YubiKey and an Android phone with NFC then the Yubico Authenticator is pretty awesome. You can store your OTP tokens on your YubiKey. Also open source.
I would not recommend using “ufw” for configuring the firewall on a production server (as it’s not easily composable and lacks configurability for more complex rules). I can recommend “ferm” instead as it allows you to compose multiple config files (important for automation e.g. when using Ansible) and allows describing more complex rules that e.g. involve policies.
How would you define a production server? Do you mean in the context of a home server or a server used by a large company? The guide is intended for a server for home use. I hope anyone securing a large corporate server is not using information on GitHub. If they are then the company has far bigger problems than security.
I think for home use, ufw is probably good enough. I've been using it for 3+ years and it's worked out okay for me okay.
I have not heard of ferm but I will check it out. Thanks!
Sorry, very late reply: We deploy all our infrastructure with Ansible, hence we want to have a way to configure the firewall for each role individually without overwriting previous configuration. For example, we have firewall rules for IP-Sec connections, SSH connections from the bastion host and then specific configurations for applications like databases or message queues. With ferm we can just create individual configs for each of these and put them in a directory where they are loaded sequentially and automatically. This allows us to iteratively define firewall rules and deploy different Ansible roles.
For a home server ufw is probably good enough, as I said I wouldn't recommend it for "serious" use in a highly automated environment.
What I feel like all the responses missed is that while you're right in that a password-protected private key provides a second factor... what is typically meant by 2FA is two factors checked by the server. The server has no way to know you're using a password-protected key so it shouldn't really be regarded as a factor.
You are incorrect. The only factor used there is the key. The password simply decrypts the key to perform the requested authentication.
2FA would be private key (password-protected) and a separate (most likely one time use) password. Something you have (key that you decrypted) and something you know (the one-time password).
Keys are easily (and persistently) added to ssh-agent, at which point it is easy to forward and will generally silently authorize without any further user interaction. That reduces it to a single factor.
Compare that to a totp challenge or a yubi key plus a password. Having one of them won’t get you the other.
If the private key cannot be read by anyone and the user must authenticate to get the device to sign the authentication request, then it could be considered 2 factor by some interpretations.
Always. Google gives me 5 thousand pages of ads before the results I want. I hate to admit that sometimes I still use Google for searching. Like I love their shopping search.
It would also be proper to disable non root access to /proc among other things. You can do that by simply mounting with hidepid=2 or adding it to fstab.
Why? Disabling access to /proc will disable a huge number of useful features, such as the ability of a process to monitor and manage its memory usage, to debug itself, and so forth.
Dont forget about namespaces which lets you isolate apps. and Apparmor to restrict file and network access. And also chroot and setuid to drop privileges and change the root path. And unix sockets / named pipes. Instead of letting your daemons access the whole network have them listen on a unix socket.
And scan your own network, both from the outside and inside.
I am not familiar with namespaces in this context? Can you please point me in the right direction? I have added AppArmor to my to-do list. I know what chroot and setuid are, not sure what to mention about them in the guide? And I will look into unix sockets.
Can you recommend any good network scanning tools? I've been on the hunt for a good one.
I think the most useful for sysadmins are the network namespace that lets you put an app into it's own network, so it can not see the rest of the network. Container technology makes use of namespaces to create lightweight VM/containers. See man namespaces, lxc
nmap is a popular tool for network scanning. I've also found tcpdump to be useful for looking at network traffic.
setuid and chroot are useful for programmers, so once the app is up and running, it can chroot into a data-dir and drop root privileges using setuid to a unprivileged user. As a sysadmin you can also start the app from within a chroot and run it as a
unprivileged user which is preferably. Most (free)BSD tutorials go though setting up a chroot jail, it's not as common in Linux.
Containers, VM's, and chroot will not stop a very determined attacker, but the more restrictions the harder it will get.
Security is applied in layers: First you want to prevent people from the outside. Access is most often gained through exploiting some service/daemon/app running on the server. So the app should have as little privileges as possible.
System access is often then gained by exploiting another app (so you want all apps to be locked down, not just the network facing ones), like getting Apache to run curl, that sends a internal request to another app that has a known vulnerability and happens to run as root. (nobody thought that app not accessible from the outside needed to get patched).
Once the attacker is inside the server, you also want to prevent access to other computers in the network.
Securing ssh, 2FA for ssh, using key authentication...
Yes but first and foremost you don't expose ssh to all internet, it should be allowed only from known IP addresses and you should VPN to have connect to that known addresses. Section about firewall config just tells to open ssh...
It might help protect against an OpenSSH zero-day but those are pretty rare.
I've been running my home server with an completely open SSH port since 2012 and haven't been hacked yet. And I know since I check occasionally with Tripwire and/or RPM verification from a clean boot.
The only downside to an open SSH port is the thousands and thousands of log spams from connection attempts. The upside is that I can log in from anywhere using my phone directly or as a hotspot for the laptop.
Oh yeah, and VPN just pushes the problem back one level. For me SSH is the VPN. Requiring VPN access first to get to SSH is just pushing any security problems back into the VPN server. Because now IT is the one with the open port to the internet.
Can you use Tripwire for free for home/consumer stuff? And what does RPM verification do? Just make sure all the packages you have installed are legit?
Tripwire is packaged up in Fedora. It's listed as GPLv2 licensed. So, yes.
RPM verification checks that all of the files installed through RPM have checksums that match the original RPM. It also checks that the RPM cryptographic signatures match.
So that should guarantee that files like the kernel, systemd, /bin/sh, /lib/libc.so.6, etc are not compromised.
A system can still be vulnerable to persistent attacks installed in unwatched files such as /root/.bash_profile, /etc/profile.d, extra files in /usr/systemd/system, etc. So you also have to check for extra files that you didn't install.
I don't have anything except Secure Boot to protect against UEFI attacks.
That is the thing, if we are talking about my personal linux server I am good with key auth + fail2ban and ssh on high port to have less spam in logs.
For work related stuff if I have to deal with multiple people accessing multiple servers then it is different story. Strictly forbidding ssh to have people connect from one IP is a lot more control. Then also key auth but I don't have to setup some google auth.
if your primary goal just is to hide SSH then you could have it on a none-standard port and enable port-knocking. Then you’re reducing your visibility to network traces.
Ultimately though, many would argue that’s just security through obscurity so you’d still want something like fail2ban or denyhosts running - namely a tool that monitors your log files for failed log in attempts (or other suspicious activity) and then auto blacklist that IP in your filewall. I’d personally recommend fail2ban over denyhosts for a variety of reasons but ultimately either is better than none.
What I also like to do is have the public SSH box a bastion server with its own unique credentials so you effectively have the same multi-tiered authentication as you would for VPN.
You can also add MFA for SSH too if you wanted. In fact there are a few PAM modules you can use; from captchya’s (to reduce bot effectiveness) to Google Authenticator.
Even enhanced SSH tools like the file server protocol (SFTP / scp / etc) and port forwarding can be enabled or disabled for specific logins / groups or everybody if you wanted.
So it’s entirely possible to harden SSH in the same way you would VPN.
I have a need to be able to SSH to my server from unknown sources, like my friends house or something. And VPN just moves changes the open port so no real benefit there. I agree on the firewall...
There isn’t lots of ways to do it; there’s just the one: firewall rules. There just happens to be lots of different types of filewalls.
If you want to get clever then you can enable port knocking but my personal preference is just good old fashioned whitelist of IPs with fail2ban running ready to auto-blacklist any of those IPs that have too many failed login attempts in a given period of time.
Does anyone have experience with deborphan? Not previously aware of it, I installed it, and it reported 2 programs which I use very frequently (conky and cryptsetup) as being orphans. I dare not run the uninstall script in the linked guide.
Would you mind pasting the output of `deborphan` showing those two or creating a new issue on my GitHub page? I want to show it to the author of `deborphan` to see if he knows why its happening.
There is a balance to be had here. In a high volume, sensitive environment you'd run your own mail server. But that is a challenge in-and-of itself. I don't recommend it for a home setup because there are numerous things to worry about like MX forwarding and what happens if your server goes down -- where does the e-mail go? Granted, using Gmail means if your server is compromised the bad-actor would have your gmail password which is why I recommend enabling 2FA/MFA on Gmail and then using an app password. Check http://reddit.com/r/selfhosted if you want to self-host mail.
It's better than running your own smtp servers. But the article proposes using your own gmail account, to which the server has an app password. So a server compromise leads to your own email being compromised. It should be a separate gmail account.
This is my first time putting a guide like this together. I see a lot of really great feedback and I will be incorporating them into future updates.
I don't have time today but I will reply individually to all the comments that I can.
I appreciate any/all feedback/advice. If possible, could future issues be submitted to GitHub because it is easier for me to manage if everything is in one place. https://github.com/imthenachoman/How-To-Secure-A-Linux-Serve...
But right now I need to find all the other places this guide is linked to with comments so I can address them too. :)