After Action Report: Project Greensleeves

Wait... What's this?

“After Action Reports” are my tales from the front lines of building computer systems to support my pojects and interests. Lots of people build computers, and most of them do it better than me - when it comes to my screwdriver-fu, I am about average. But if you’re interested in the technology, the business, and the marketplace, especially the used market, and looking to build or upgrade with long-term value in mind, then yeah, I’m your guy.

I am also 100% independent, and I have no fear of failure - if I don’t like something, you’ll hear about it, and if I screw something up, you’ll hear about that too. If that sounds like the sort of thing you would like to read then you are in the right place.

Upgrading the Dev Server

My plan was to upgrade my dev server, which tells you two things: I have another server and neither server is adequate for my current software development requirements. My other server is based on my favorite tech purchase of all time, my Fractal Node 804 case, which is a micro ATX case shaped like a cube that holds eight 3.5“ hard drives. It stores all of my files, including my media files, and thanks to FreeBSD runs like a champ.

When I first got serious about building web applications a few years ago, I started out using this server, but soon moved to a separate dev server for two reasons. The first was consistency - running a benchmark on a system that is serving files or transcoding means it is very difficult to get an accurate picture of a program’s performance. If one of my test programs ran for 20% longer than it did previously, I didn’t know whether it was because of a change that I made, or if the server was just under high load at that moment.

The second was credibility. When you are an entrepreneur, if you want people to take your new business seriously, that process has to start with you, because people will generally follow your lead. Write all your code on the same laptop you watch Netflix in bed with, and that says one thing. Put it on a dedicated high-end machine, and that says that you’re serious about what you are trying to accomplish.

Incidentally, I didn’t learn this from business school or my time in management consulting. I learned this from my mother who runs an astrology business, creating charts for clients - mostly ex-hippies. Whatever your feelings on astrology, she always treats her business professionally, setting rates, hitting deadlines, blocking off specific working hours, etc, so her friends and family, myself included, also treat it professionally.

My high-end dev server was fairly inexpensive, since buying and building for value is my specialty. I used an old ATX case and power supply, and was able to pick up an i7-10700 for $107, which was quite a good value. I also bought a used ATX MSI motherboard for about $90, used a spare Intel 512GB NVME SSD for a main drive, and a Samsung SATA SSD for backups, and was good to go.

But after it was assembled, I found a defect - a faint, flickering green band across my monitor. It sounds like a problem with the monitor or the cable, but it wasn’t - it disappeared when I installed an inexpensive video card, bypassing the motherboard’s HDMI port. For a server, the defect made absolutely no difference whatsoever, since the system wasn’t even going to be plugged into a monitor. But it could be a sign of a failing motherboard.

I was also having second thoughts about my ATX motherboard and case - I like being able to move my hardware around between my various systems, but at that point all of my other systems were either mini ITX, micro ATX, or NUCs, and having an ATX motherboard and case hanging around seemed like one unwelcome form factor too many. If there was a chance that my motherboard wasn’t going to last, maybe I could move the dev server to a micro atx platform.

There was also one more change - I had a new SSD I wanted to install. If you haven’t been following the SSD market, there was a big jump in performance a couple of years ago where SSDs literally got 5x faster overnight. This specific SSD, a SK Hynix Gold, wasn’t quite on the cutting edge (that would be the platinum) but was still apparently capable of 500K IOPS, and I wanted to see that in action. SSD prices have also been dropping like crazy, making this a good time to upgrade.

For the motherboard, I picked a B460M DS3H from Gigabyte - paid all of $65 from Newegg on Ebay. Gigabyte is a reliable supplier and about 80% of my boards come from the company. I don’t much like the gamer aesthetic, and I appreciate not having to pay for it.

Now let’s talk about the case. One of the benefits of building your own computers is that you can buy a great case and hold on to it for many years, like I have done with the Node 804. I give that advice to people all the time, and I follow it myself, which begs the question of how on earth I ended up using a $37 case for this build. And I’m not sure that I have a good answer.

The case is the Q300L from Cooler Master, and it was on sale from Amazon Warehouse for $37, a full $2 off its list price. Cooler Master is one of those brands that I have always been completely indifferent to until recently. What changed is their release of the NR200, a fantastic mini-ITX case with design and features borrowed from much more expensive cases. I can’t recommend this case enough to any mini ITX builder.

And sometimes buying something cheap is a risk that pays off. For example, I have been using ID-COOL CPU coolers in all my systems, and that process started with a $24 CPU cooler that I wasn’t too sure about and was fully prepared to throw away, and yet it ended up being a great piece of hardware.

Perhaps this $37 case would jump start a new era of cheap micro atx builds for me. And if not, it would make for some funny subtitles at least - “the best micro atx case in its price point”.

When I started to put the parts together, it wasn’t immediately clear how to attach the power supply to the case so I went on youtube to look at reviews and builds. Right away, I saw at least two videos that said “DO NOT BUY THIS CASE”.

That’s when I asked myself if I read any reviews at all before I bought the case. Did I research it on PC Partpicker? Now I couldn’t remember if I had skipped over even that minimal due diligence.

I then found a video review from someone in Singapore, with their “English, but not as we know it” accent, and gave it a listen. I spent almost two years in Singapore, and one thing the folks there know is computer hardware. I watched him install the power supply, screw in the motherboard standoffs, flip around the control panel — wait, standoffs?

If you don’t know what a motherboard standoff is, you are lucky - they are basically holders for motherboard screws that you have to screw into the case itself that keep the bottom of the board from touching the case and shorting out. My ancient ATX case had standoffs and they were a pain to work with. But other than that, I hadn’t seen a motherboard standoff in years - every case had raised screw-holes that made them unnecessary.

The presence of standoffs made this case impossible to recommend to a starting builder. The problem is that when you try to loosen your motherboard screws, the standoff loosens too. So you either get a motherboard that’s not properly attached to the case, or a screw that’s stuck in the standoff. And even if you are an experienced builder, there are still a lot better ways to spend your time than messing with standoffs.

The case also was designed in such a way that I couldn’t get my big screwdriver to turn either the motherboard standoffs or the screws closest to the side of the case. I then tried my smaller screwdriver, and that wouldn’t fit either. So now I didn’t have all that much confidence that those screws were tightened firmly or that the standoffs would even hold, but I wasn’t giving up yet.

People who think building computers is difficult are wrong, and people who think it is easy are wrong. Conceptually, the process is simple - twist a few screws, attach some cables, and voila, we’re playing Counterstrike. But the problem is always that some of the screws and cables need to go into places that you can reach or can barely even see. It’s sort of like trying to unhook your date’s bra strap in the dark with a pair of chopsticks.

Still, at a certain point sheer determination is what gets these builds done, and this was no different. I plugged everything in to my KVM switch and turned it on, and the system booted right up, which is always a good start. I saw the BIOS screen and — uh oh, that looks like a green flickering band. Yep, brand new server, same old green band across my screen.

It was time for the proper troubleshooting that I clearly skipped the last time that I ran into Mr. Greensleeves. What did the two systems displaying this problem have in common? At this point, only the CPU, the RAM, and the power supply. Time to swap parts and see which part was defective or failing.

I had a spare power supply and spare RAM, but I didn’t have a space 10th gen Intel CPU, at least not one that I was willing to pull. The i5-10600 in my gaming system was capped with an Asetek water cooler, and it wasn’t one I could remove easily.

I went ahead with the power supply and RAM swap, eliminating those as possibilities, and what was left was my CPU, so yes it looks like it has some defect that affects video output. That’s the risk you take buying on Ebay - getting parts that are defective, but the defect isn’t something you understand or spot right away. But on the other hand, the CPU had been performing great - it was just its resale value that was impaired. If that was the only problem, I could live with it.

I prepared my FreeBSD USB installer, and found the second problem, and it was a big one. The USB stick didn’t actually fit into my new system, at least not on the left side of the I/O shield. How was that even possible?

After a few minutes of investigating, I had my answer, and it was bad. My motherboard wasn’t lying flat. Instead, it was at a slight angle, which threw off the alignment of the IO shield at the back and made it impossible to plug in the USB stick. Maybe it was those standoffs, or maybe it was just a poorly manufactured case. Either way, it wasn’t good.

I took the board out of the case, and then put it back in, and it was better, but not by much. “I am returning this damn case”, I said to Amazon. Because I don’t have Alexa, I was just yelling the words out my window and hoping they reached Jeff Bezos.

Keeping the case wasn’t an option since it was literally bending the board, and who knows what issues that could create over time. I had two options, first, either putting everything back in my old ATX case and pretending like this evening never happened and that this was just the longest and most convoluted SSD replacement in history, or second, I could try my backup micro ATX case.

My backup micro atx case is a red low profile box that takes an SFX power supply. If that sounds strange, yes, it is a strange case, and I am not even sure who makes it. I bought it many years ago - maybe 2011 - for a media PC, and for a few years it housed a Core 2 Duo, a small low profile video card with HDMI output, and a 2TB drive filled to the brim in the days before they were cracking down on The Pirate Bay.

It wasn’t much of a case, but it had a size advantage over just about every other micro case, which meant that I could keep this server on a shelf somewhere out of the way. Which was impressive, given the specs of the server - 8-core/16 thread CPU, 64GB RAM and an ultra-fast 1TB SSD. And I could also use one my SFX power supplies to power it, which inexplicably, I have about ten of.

However, the CPU fan & heat sink definitely wouldn’t fit - a beefy Scythe Fuma clone from ID-Cool - so I looked in my box of old CPU fans & heat sinks and came up with a low-profile Silverstone that I remembered using for an i5-6600K in simpler times.

It had a great chrome aesthetic, but its chances of cooling an i7-10700 running at full blast seemed pretty slim. But maybe it would work temporarily and I could pick up a new low-profile cooler, which was something that I had been meaning to do anyway for this exact type of situation.

The system booted up and made that horrible “fan-meets-wire” noise, but after a few cable twists, it was good to go. I’m generally not one to build for aesthetics, but now I had a small bookshelf-type system that was seriously powerful and small enough to stick in a corner and was pleased with it. Which meant that now it was time to see how bad that Silverstone CPU fan & heat sink really was.

I ran my latest Slipcache benchmark (Slipcache article coming soon), getting the exact time I expected to. I checked the CPU temp and it read 57C - holding up well. Since my benchmark is single threaded, I then ran 16 copies in parallel to use all of the cores/threads, and sure enough:

Coretemp2: critical temperature detected, suggest system shutdown

And that was about what I expected. Silverstone’s CPU fan was made for aesthetics rather than actually cooling down CPUs. But in fairness to them, when this fan was manufactured, CPUs didn’t get nearly as hot as they do now. Intel 8th gen changed the game, making anything less than a full tower cooler with twin fans completely inadequate for an 8700, 9700, or 9900.

That wasn’t something I was going to fix in this session, but there was a simple workaround - turning off Turbo in the system BIOS. The immediate effect in my cache read benchmark was brutal - the system was now running 35% slower. But a system that runs consistently slower is still consistent, which was my goal in having a dev server in the first place. As I made changes to my programs, I would still be able to determine if my change had sped up the code or slowed it down.

And that meant that I was done for now, waiting on a new low profile CPU heat sink & fan so I could return the system to its intended speed.

What I learned:

• Read the reviews before you buy something that you plan to use, no matter how cheap it is.
• Having spares on hand can be the difference between being able to get something done and getting nothing done in a particular session.
• Even if you’re absolutely sure a defect is in the motherboard, the problem could still be the CPU. You’ll need a full swap of each to be sure.

Addendum:

And now I can run my dev server at full speed again. The low-profile cooler that worked for me was ID-COOLING’s IS-55 ARB, and it cost all of $40 from Amazon, and that’s with California’s merciless 9.75% sales tax, so I consider it an excellent value. If you’re in a similar situation, I highly recommend it.