An orange in an apple orchard
Best way to remember that Stark’s CMOS battery needs replacing: plugin, let’em charge, boot up, oh hey BitLocker.
On the upside being anal retentive about such things, it was more of a pain in the arse to input my recovery key and decrypt the system drive than to actually find where I had encrypted that 😁
There’s several upsides on standardizing on cables and devices when possible. In my case, that’s been braided (i.e., tangle free) USB-C cables rated for 100W charging when the cables are long and comparable 10 Gbit/s or faster rated cables when they’re short.
One of these upsides is “Ahh, it’ll charge a laptop!” when paring a suitable charger with any of my longer cables. These cables are usually poor on data speed but superb at power delivery, which is often what I want when the desired cable is measured in meters, which is also when I really want tangle free….lol.
Another is knowing that when I grab a smaller cable, it’s going to be good enough to feed I/O devices like a NVMe based SSD or any SATA thing I’ve still got handy. Aptly, most of these short cables either came with NVMe enclosures rated for 10 Gbit/s USB connectivity or are in fact Thunderbolt 3/4 cables rated for both 40 Gbit/s connectivity and 100W charging.
Increasingly, when the cables are short I’m aiming for 40 Gbit/s + 100W unless they’re packaged with something. The downside is that Thunderbolt cables are costly and have limited cable lengths, but generally are sufficient for ‘all the USB things’ once you’ve groaned at the bill. If I find myself buying a short cable these days, I’ll save up for a Thunderbolt for future proofing because more and more of my devices support either Thunderbolt or USB at 40 Gbit/s.
For devices in general, I’ve been swinging for USB-C 10 Gbit/s for a while now. Things like motherboards, drive enclosures and external drives, USB hubs and PCI-E expansion cards are chosen based on this. This choice was made based on the rise of the NVMe external drive, and the fact that such a cable will be no problemo when pared with my older gear that maxes out at USB 3.0 or SATA speeds.
Similarly for chargers, the rare time that I buy a charger, I’ve generally aimed for the 90~100W scenario. In the sense that most of my devices will happily charge from a 45W or 65W charger, and the hungriest ship with a 90W charger.
Is this excessive? Not really. Why? Well, let’s see… my primary machine has 40G ports, my gaming machine has 10G ports and a card with 40G ports. SteamDeck has a 10G port and my file server has an expansion card with 10G ports.
Much like USB-A and MicroUSB-B has become relegated to specialized and rare things around here over the past decade, so has 5 Gbit/s connectivity begun to age out of the herd ;).
Annoying factoid, the modern naming of Linux network interface cards provides a consistent way of naming the devices. But not a permanent one.
Taking advantage of Rimuru’s old dual port 10G USB-C card that I replaced with a Thunderbolt 4 card, happily works. But as a side effect this means that enp3s0 and wlp4s0 are now enp4s0 and wlp5s0, which as you might expect breaks the networking configuration for both interfaces.
Because why the fuck would you expect devices to retain their topology just because they happened to be soldered to the board? On the upside at least it became obvious what was going on when I inspected the files in /etc/NetworkManager/system-connections and noticed that the digits had changed.
I’m guessing that since Zeta’s lone PCI-E slot is an x16, it ends up numero uno. It just so happens that I have an PCI-E x4 card plugged in with a USB controller instead of a GPU, because the machine’s jobs are all server related. Although, I bet she would make a dandy little gaming box in as much as a 2-slot wide GPU and a SFX PSU can actually handle anything modern and juicy. Especially if Valve was to you know, release StreamOS 3.x for PC instead of Steam Deck only 🙂
lspci -t -v
-[0000:00]-+-00.0 Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne Root Complex
+-00.2 Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne IOMMU
+-01.0 Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge
+-01.1-[01]----00.0 ASMedia Technology Inc. ASM1142 USB 3.1 Host Controller
+-02.0 Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge
+-02.1-[02-05]--+-00.0 Advanced Micro Devices, Inc. [AMD] 500 Series Chipset USB 3.1 XHCI Controller
| +-00.1 Advanced Micro Devices, Inc. [AMD] 500 Series Chipset SATA Controller
| \-00.2-[03-05]--+-00.0-[04]----00.0 Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
| \-01.0-[05]----00.0 Intel Corporation Dual Band Wireless-AC 3168NGW [Stone Peak]
+-08.0 Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge
+-08.1-[06]--+-00.0 Advanced Micro Devices, Inc. [AMD/ATI] Cezanne [Radeon Vega Series / Radeon Vega Mobile Series]
| +-00.1 Advanced Micro Devices, Inc. [AMD/ATI] Renoir Radeon High Definition Audio Controller
| +-00.2 Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) Platform Security Processor
| +-00.3 Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne USB 3.1
| +-00.4 Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne USB 3.1
| \-00.6 Advanced Micro Devices, Inc. [AMD] Family 17h/19h HD Audio Controller
+-14.0 Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller
+-14.3 Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge
+-18.0 Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 0
+-18.1 Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 1
+-18.2 Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 2
+-18.3 Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 3
+-18.4 Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 4
+-18.5 Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 5
+-18.6 Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 6
\-18.7 Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 7Now that Zeta is effectively operational, I’ve turned my master plan to its next stage: not losing data.
Cream’s drives had a very simple arrangement. One drive, designated “Master”, or M:, was the base of all the file shares kept there. A second drive, “Backups”, or Z:, was kept next to it, and a scheduled task would run a robocopy in mirror mode three times a week to sync the drives up. Nice and simple and cheesy, and for bonus points the mirrored drive racked up about half the power on hours over the past few years. For monitoring purposes, the log was saved to one of Cream’s internal drives which was periodically imaged to the host specific backups area on the master drive.
Zeta on the other hand doesn’t have the curse of NT, but I did kind of like this simplicity. It fits my recovery model where having an easily recovered copy of data is desirable, but changes are infrequent enough that rolling up the backups every 2 or 3 days is probably okay. At first, I decided to create one disk as ext4 to function as the backups, because dependable and trusted, while making the other xfs to function as the master, because that’s the default in AlmaLinux 9.
This created one small problem however, in that getting rsync to play nice with the SELinux, POSIX ACLs, and a few extended attributes proved to be a pain in my ass! For SELinux, you can just relabel the drive after. Not something I want to scale up to 8 TB but not too bad for the actual storage use (2 TB) today. But then we’ve got the issue of the POSIX ACLs and extended attributes used on my file share infrastructure.
Turns out that rsync’s --archive flag effectively breaks the flags you would want for synchronizing these, and then leaves you to go fiddle around with permission masks. So, I said fuck that. I was rather disappointed in rsync at that, but let’s face it, acls and xattrs aren’t that popular when 1970s unix permissions are an 80% solution.
After taking suitable backups (one local, one remote) of the critical files, I set about turning to tools that I know how to fuck with. The backup drive was sacrificed to create one disk of a RAID1 Mirror, and since madam allows specifying the drives like missing /dev/sdwhatever or vice versa, it was easy to spawn the array in degraded form. Then sync the data to the array from the master drive, before wiping and adding it to the mirror’s missing slot. About 10 or 11 hours later syncing at max speed, everyone’s all riled up and gone through the reboot test.
How did I migrate the data if rsync was being a bugger, you ask? Well, it’s slow as hell, but cp --archive and tar --acls --selinux --xattrs really does do what you want when you’re Rooty Tooty and want a lossless copy :P.
In the past, I would typically have used LVM2 pools to manage this sort of operation. It’s overly complicated command line administrata, but hey, it works well and it has features I like, such as snapshots and storage pools. The advantage for me of mdadm is that it is very simple to manage thanks to fewer moving parts.
Having been “That guy” at some point in my career who ended up writing the management software my old job used for mdadm software raid in their audio IRDs, and then later extended to custom hardware built ontop of firmware raid, I know how to use mdadm and more importantly, how reliable it is — and how easy it is to recover a mirror without fucking up. Which, you know, is like the number one way your data goes bye, bye when recovering, right next to oh shit the drive died before it was synced. As much as I appreciate LVM2, it’s got enough moving parts that I’m more leery about the failure scenarios. More importantly, I have more experience with mdadm failure and recovery than I do with LVM.
Of course, this does create a new problem and its own solution. Since my backup drive is now in hot sync with the master drive, it is no longer uber idle enough to be considered a ‘backup’. No, it’s redundancy to buy time to replace drives before the entire array goes to the scrap yard.
This doesn’t really change my original recovery scenario: which is “Go buy two drives if one fails,” it just means that there is a higher probability that both drives will actually fail closer together when that happens. What’s the solution to this? Why, my favorite rule of data storage: ALWAYS HAVE A BACKUP! Thus, a third drive will be entering the picture upon which to do periodic backups of the array, and be kept separate and offline when not being refreshed.
In practice though, this will be more like a fourth drive; in the sense of ‘smaller disk, most important data’ and ‘big ass disk, all the data’. My spare archive drives are large enough to easily do the former, and one can basically contain the entire ‘in use’ storage or close to it, but none of my spares for sporadic backup has the capacity to handle the entire array.
Having finally merged some code that’s been stuck in my craw, I decided on a mini-celebration: pizza and eggplant parmigiana, although sadly I forgot about the beer in the fridge. Oh, well; it’ll be there to go with the leftovers 😋.
On the flip side, I think it’s almost time to declare Zeta an operational battle station.
The first problem was I/O performance. Her predecessor, Cream had been pressed into sharing its Wi-FI with Rimuru, leaving the SMB shares on Cream only accessible via wireless clients. Having fished out the aerials that came with Rimuru’s Motherboard 2.0, that solved that connectivity gotcha. But not the simple fact that the file server and the clients are within a meter or two of each other, and the access point is across the house! As much as I suspect a mesh system will be the upgrade path for my network, I’m not replacing that router until it dies or Wi-Fi 7 is ready to rock.
Thus, my shiny new file server was only achieving about 5 MB/s connectivity with my Mac and PC on the other side of the L-shaped monster. Now, I’ve never expected big things of Samba compared to NT’s SMB stack, but Samba’s got waaaay better performance than that and so does Zeta’s hand me down platter drives. My solution to this problem? Gigabit!
At first, I attempted to solve this problem using the combination of libvirt and pfSense. But, I didn’t have much luck getting the bridging to work in order to have a VM on the host be a router while the client functions as the physical. In the end, I discarded this idea and configured Zeta to function as the router for my little local IPv6 network. Yeah, that’s right: I said IPv6, baby! Since this is a local network intended to join Zeta (server), Shion (Mac), and Rimuru (PC) and the occasional other machine, I opted to set this up as IPv6. There’s no real need for IPv4 in my desk’s wired LAN. Maybe I’ll enable IPv4, so I can jack old PowerBook G3 into the switch since MacOS 9.x probably lacks IPv6 support the way Sonoma lacks AppleTalk support 🤣.
Configuring things was pretty easy. A little bit of radvd to enable the Router Advertisement and Router Solicitation issues and for good measure, setup DHCPv6 as an insurance policy, and configured the Ethernet port with the desired address and itself as the gateway. In the future, I may try setting up BIND, so I can have DNS A records map to Zeta’s IPv4 address on the household Wi-Fi and AAAA records map to Zeta’s IPv6 on the desk’s Ethernet, or perhaps even separate domains. But I’m a little hesitant of taking out DNS whenever I reboot the server.
On the flip side, thanks to the lack of fuckwittery, Samba and the SMB stacks on Mac and NT just handles this case fine. Navigating to \\ZETA or smb://ZETA while jacked into the local Ethernet switch nets me about 80 to 115 MB/s, or roughly how fast you can spew data over a Gigabit link to SATA powered things. Seems that the SMB stacks are smart enough to prefer the local Ethernet, but something more DNS aware is how to fix cases like SSH.
The next phase has been setting up the virtual machine environment, which will probably replace the Parallel’s VMs I sometimes spin up on my Mac and the WSL2 environments on my PC. For this, it basically amounted to setting up a bridge interface with the same IP information and using Zeta’s Ethernet port as its bridge port. Then setting the virtual machine’s second interface to bridge to LAN, so that it can be routable over the local switch.
Thus, Shion, Rimuru -> Zeta works. Shion, Rimuru, Zeta -> some VM on Zeta works. Muhuahuaha!
As the process of migrating files from Cream to Zeta continues, and rather devolves into making more like 1983 than 2023, I am reminded of how much I despise using Windows machines in important roles on my network.
Yes, the whole experiment of using Windows 10 for my home file server worked out pretty well relative to what I expected. But also, yes–it has pissed me off a lot over the years.
More than a few times in the last 6 – 8 years or however long it has been, I’ve thought to myself, “Gee, if I had just loaded Debian or FreeBSD a few months later like I had planned…” that it would have been cheaper in the long run. To be fair, there have also been times that I found it rather neat, but most of those involved things like ssh/scp becoming (mostly) first class citizens in the land of NT.
I am sure, whether or not Zeta proves to be closer to the “Ten year server” plan than Cream did, AlmaLinux will at least be less of a pain in my ass than NT was.
And this my friends, is why I love having extra memory!
[terrypoulin@zeta ~]$ dd if=/dev/zero of=./dd.test bs=1M count=2000 2000+0 records in 2000+0 records out 2097152000 bytes (2.1 GB, 2.0 GiB) copied, 0.38554 s, 5.4 GB/s [terrypoulin@zeta ~]$ dd if=/dev/zero of=./dd.test2 bs=1M count=2000 oflag=direct 2000+0 records in 2000+0 records out 2097152000 bytes (2.1 GB, 2.0 GiB) copied, 4.01215 s, 523 MB/s [terrypoulin@zeta ~]
This machine has a cheapo 1 TB Inland Professional 2.5″ SATA SSD to serve as its system disk. But she’s got 64 freaking gigs of RAM. Yes, that’s right – sixty-four freaking gigs!
[terrypoulin@zeta ~]$ free -h
total used free shared buff/cache available
Mem: 61Gi 966Mi 57Gi 8.0Mi 4.1Gi 60Gi
Swap: 31Gi 0B 31Gi
The first dd command writes 2 GB of zeros to a file one MB at a time, as fast as the system can go. Thanks to the OS being able to say, “Hey, I’ve got memory to buffer that; carry on wayward son,” it is completed Really Damn Fast. This buffering isn’t good for the case of a slow removable disk (or IMHO, oh shit, batteries), but is very effective when doing a lot of file I/O such as compiling software or working on large projects with many files. By contrast if the system had little available memory, it wouldn’t go so fast.
The second command, effectively says the same thing but uses Direct I/O to ensure the data is spewed the disk quickly and immediately, meaning that we get the speed a decent SATA SSD can achieve when combined with its own little bit of internal buffering. But we don’t experience the crazy speeeeeeeed that is RAM.
Why is this important for Zeta? Well, Zeta is replacing Cream — which means she has an 8 TB storage array to take care of, spends most of her life dealing with networked file transfers, streaming media to client machines, and unlike Cream, will end up running several virtual machines thanks to having enough extra memory. Did I mention, fire sales on DDR4 have already begun to make way for DDR5? 😁
For the most part, the high point that Windows NT has achieved for me is “Not pissing me off by default” and becoming a fairly decent shell upon which to use the Linux things that I care about without needing a second machine or dual boot, thanks to WSL. The era of Windows 10 also brought iterative improvements to system components I care about like the command line environment. But versus native Linux, the main win for me is better access to DirectX games and Microsoft’s office apps.
But truth is, there are certain parts of Windows that are likely to always piss me off. Namely Bluetooth support, and to a somewhat lesser extent anything related to USB or networking will inevitably drive me nuts given enough time around NT.
Thinking about this as I finish up a few things before bed, I realize I typically like using MacOS. The aspects that piss me off tend to revolve around muscle memory, like how some common PC shortcuts are cmd+key and others are ctrl+key. Which are shell level uniquenesses not systemic design. On that note, I’ll add that I tend to find iOS/iPadOS rather more meh, or average than pleasant.
By contrast things that irk me so about using modern Linux as a desktop are the quality of mail clients, lol.
Somehow the joys that are using Cream’s internet connection sharing to power Rimuru’s networking requirements, is reminding me that converting to a mesh network is one of the cheaper ideas on my long term planning board.
Dang gum computers (>.<)
Decided to take a break from my expedition into Subnatica’s Lost River and break out my old PowerBook Duos. I must have a strange concept of fun.
I’m a little saddened that the Duo 2300c has gone to smear and physical rubber band land while it’s been in storage. But I now know a few things about how it boots. Thanks to Shion unexpectedly being able to mount its drive, I know the system doesn’t boot because the 8.6 install files from Apple’s anthology repo are looking for 8.5 files. Nuts. On the flip side I also learned that plugging in my Duo 230’s drive, that it will boot on the 2300c. Actually runs pretty nice despite it all being emulated, since all the applications on that drive are either for a Motorola 68030 or older, or a fat binary for both.
Given the state of the screen, even if it has a snazzy PowerPC, I opted to swap my memory cards back so that the 230 has 24M of RAM and the 2300c probably has 8M or whatever. At first, after seeing the state of the screen, I decided to try seeing if the touchpad could be swapped. The 2300c and 230 top covers only differ in the labelling and having a trackpad mounted where the cutout for the trackball would go. Sadly, I wasn’t about to use the trackpad, so I swapped these back. I don’t mind the 230’s trackball, so much as it tends to stick or stop sensing the ball rolling no matter what I do to the darn thing. Probably should just buy a Wombat to connect a USB mouse to it.
Interestingly, I also learned that the 2300c mainboard has both its new IDE header and an old SCSI header in exactly the same place as the earlier 230 and its relations. The machines are really, really similar. I’m pretty sure that the only major differences are adapting the chipset to accommodate a PowerPC instead of a Motorola processor. Since the screen is going bad and its keyboard sticks less than the 1992 one, I decided to at least switch out the keyboards so that the 230 has a better typing experience.