All right. Uh, bookend for the, uh, recorder.
Missed second.
Is being recorded.
Everything you say can and will be used against you. Quote, public opinion.
Alright, we're recording. Okay, Sam, you're up. Alrighty. Okay.
I'm up.
Uh, welcome, everyone. This is the bookend for the recording. This is the St. Louis Linux User Group Meeting of, uh.
Uh, Thursday, August the 21st, 2025. Uh, welcome, and our speaker tonight will be Stephen Lombard.
And he's going to be telling us. Pardon me? Oh, hello.
And, uh, he's gonna be telling us all about logical volume managers, and why they're logical.
So, uh, before we do that, uh, just got a few, uh, odds and ends to, uh, talk about.
In terms of announcements, um… St. Louis Linux user group is part of Slug, which is the Linux and Unix user group in St. Louis.
And, uh, um… We're actually a 501c corporation.
Uh, our general meeting is always on the second Wednesday of the month.
And, uh, the, uh, 8 days later is the STL Linux, the lug meeting.
Uh, we also hold, uh, two other meetings. I'll talk about those in a minute, uh, that are.
Regularly held every month. Let's see… our webpage, uh…
One of the interesting things on our webpage, first off, is we put up the.
Jitsi or Zoom connection for. Meetings that, uh, we're holding.
Uh, secondly, our presentation archives. Ever since the COVID pandemic started.
We have managed, I think, to record every one of our meetings, and uh… So we've got a few that were recorded before that, but uh…
We try and keep them all in the archives. Um, so that was the bookend you heard at the beginning, was to start that.
So, one, uh, word of caution, uh, remember, just because it's not.
Recorded as part of our archive. If we're not recording. Somebody else can be reporting at home, and uh… so you're…
Be aware. Okay, um, our meeting tonight, uh, obviously the meeting came up about 6 o'clock.
Uh, we just started the introductions here a little bit after 6.30.
And, uh, in just a few minutes, hopefully before 6.45. We'll get to the presentation. After the presentation.
Uh, we'll, uh, uh, not just answer any questions. Pertaining to the presentation material.
But, uh, if anybody has any other… Off-topic questions they care to ask or propose.
Comments, uh, we're open to them. Next slide.
Okay, uh, introductions. Uh, I'm the chair this evening. I'm Gary Meyer. Well, let me take that back. I'm the MC this evening.
Uh, my name's Gary Meyer, I'm also the president of the group.
Uh, the chair of the LUG is James Conroy. James is, uh, uh… Uh, just been getting back from his honeymoon. He may join us tonight, or he may still be busy, I don't know.
So, uh, our, uh, our host is Lee Lambert. Uh, he's the.
Principal scientist, Omnitech Corporation. And, uh, Omnitech also provides us with a lot of technical support and posting on their systems.
He's also the treasurer of our group. Uh, most of the production stuff's being done by Stan Reichert. Stan's our vice president.
And somehow, he makes this all come together all the time, so thank you, Stan.
So, chat monitor? Anybody want to volunteer to just watch the chat and Zoom tonight and see if there's any comments that get typed in?
Crickets, crickets… okay, no volunteers, alright. Um, alright, um, I'll try and keep an eye on it. So, I'm going to ask our speaker real quickly, um.
Steven, uh, do you want people… Do you want people to interrupt you?
Feel free.
Verbally? Okay. Um, so you can either ask Question Stevens live during his talk, or.
Um, if you want, you can also type stuff into the chat, and if I notice it.
I'll, uh, I'll try and bring it to his attention, or to the group's attention.
Topic Wrangler. Not for tonight, but just for always, for all of our meetings. It's, uh, always good to have.
Uh, somebody else helped, uh, come up with new speakers, new topics to talk about.
Um, so if anyone would like to volunteer, uh. Love to have a volunteer to help me find some new speakers, and uh…
And if nobody volunteers, you're stuck with me again, so it's a good idea.
Have them available. And of course…
Yeah, we can hear you, Dolly.
But, can I be her? And I'm here… I'm here because I just want…
To, uh, meet people who also like Linux as much as I do.
You're welcome.
Oh, great! Great! Welcome! We always, always love, uh, having a new face and new voice on the system, so…
I've actually kind of been here before, but I haven't been here in a while.
So yeah. Okay, okay.
And just out of curiosity, what brought you tonight? Uh, the topic, the speaker, uh, just to idle the calendar for you?
I'm trying to get back into the community.
Okay, alright. Well, you're always welcome here, so come on back.
And, uh… Let's see, um… Oh, also a thank you to our speaker tonight, uh, Steven Lumbark.
Actually, we originally had a different speaker, different presentation lined up for tonight.
And, uh, the, uh, the guy who was putting it together.
Got tapped on the shoulder by his boss, uh, early last week, or the week before.
And said, okay, we've been transferred to a different division. So he's been kind of, uh.
Wiped out for the last week, and I called Steven. Well, and so, hey, Steven, um, you know, would you be available to talk in a couple of days?
And Steven very graciously volunteered, but I happen to know that Steven actually.
Uh, had a bunch of these slides all set and ready to go for a presentation back in December.
It's not something you just did off the spur, but he polished it up in the last few days for us, I'm sure, so… But also, about 10 years ago.
Steven did a topic… a presentation for us on logical volume manager. So, this is the 10-year anniversary talk, I guess is what we're going to call it, so… Okay, next slide.
Is there.
Come there. Announcements! Okay, uh, let's see. Five days from now, usually the fourth Tuesday, is the new Linux meeting.
Uh, the new Linux group is, uh, basically. For people who, you know, whoever gets… There's no step topic of a presentation.
But, uh, whoever… feels like they are the newest person to Linux.
Uh, they get to ask the first question, and kind of push for good answers.
And then whoever the next. Most recent convert is, gets to ask the next set of questions.
Uh, so… For those of you who feel like you're an old graybeard.
Like me. Uh, and uh, you're… it's not that you shouldn't come to this meeting, because it's really amazing.
Um, you'll get 1, 2, 3 people answering, uh, some new person's question.
And there'll be 3 completely different points of view, 3 completely different approaches.
And, uh, you actually wind up, even us who've been doing it for a while, learn things from the other.
People who come up with suggestions, so… So everybody should come to the new Linux feed.
Slack meeting. St. Louis Area Computer Club, uh, it's one of the oldest groups in St. Louis.
And, uh, it's a bit more hardware-oriented, um, and yes, we'll even talk about… Microsoft Windows.
Uh, so… But, uh, that meeting's the second…
Second Wednesday of each month? Oh, I'm sorry, I'm reading it wrong now. The first Thursday of each month, excuse me.
Uh, so… Alrighty, uh, a warning there, the October the 14th of this year, Microsoft Windows 10 reaches its end of life.
Um, you'll probably be seeing more and more stuff in the news about that.
Uh, the reason why we mention it here is because a lot of people who are currently using Microsoft Windows 10 will be tugging on your shirt sleeves soon.
Saying, how do I put Linux on my machine to replace this old, crappy Microsoft Windows?
A lot of people are using heroin, too. That doesn't mean we can't help them.
Ooh, that's right, that's right. Okay, um, as I said.
We try and keep an archive of all of our own.
Old presentations, um, so please visit our homepage, and you can shop around in the archives and see if anything.
Is interesting for you. Our calendar, uh, we… We originally kept track of not just our own meetings, but other.
Techie organizations in St. Louis. Uh, since the age of COVID, and we've been bringing in so many new users from, uh, out of.
Out in the metro area. Um… Yo.
You're all over the map tonight, Gary.
Yeah, yeah. Um, so… the… you know, we put all kinds of stuff on our calendar that just looks interesting, so…
And by all means, join our, uh, Discuss mailing list. It gives you a chance to ask questions or make comments.
Um, uh, online, and uh… Uh, between meetings.
Uh, call for help. Uh, if anybody can't wait to get their call for help or comment in until after our main speaker.
Go ahead and pose to… see if there's a… Quick, short answer, we can get you right now, if you need one up front here.
Okay, not hearing anybody. I'll presume. We can wait on Paul for Helps until after our main presentation.
With that, next slide. Bye by Windows 10, there is what I was just mentioning, there's a number of references on there.
Amazon book about this end of, uh, end of life for Microsoft Windows.
And, uh, PDF version that's available. Um, so… a lot of stuff.
This… this book by Stephen Whiteman from the Atlanta Linux Users Enthusiast Group.
Uh, half the book tells you how to… to do Windows 11, the other half tells you how to go to Linux, and it's very pretty balanced. It's, you know.
And it's available from Amazon as a paperback, or from Kindle, or whatever.
Or you can just pull up, legally, pull up… pull down the PDF of the whole book from these two different places.
He actually, in his first chapter, gives those two PDF, uh.
Links.
One suggestion, too, and it's available pretty cheaply. If you're playing around on Linux, especially if somebody's converting.
Is, uh, the Linux and Unix System Administrator's Handbook. Nimoth, sea bass, Snyder, hind, I forgot who else.
Uh, the fifth edition of that was the last one that came out.
And it's fairly old, but most of it is still perfectly relevant.
Nice. All right.
And the nice thing about his… Treatment of Linux as he… give some pretty good suggestions on how to do backup properly and things like that.
And…
Everyone I've seen that does any of this, because. Almost none of the books.
Show you… how to use LVM.
Most of the solutions they show you are idiotic.
Because once you… once you've decided to depend on the physical layer, and you're using partitions.
Uh-huh.
Basically hamstrung yourself on… any good way of controlling the storage.
So you can't snapshot it, you… I mean, all of… all of the worst…
Stupidities that I've seen with Linux. You know, the people having to use Part-Ed because they want to change the amount of space available for their.
You know, they're using for VAR. Um, all that crap.
Goes away immediately when you start using LDM. It's like using physical memory instead of virtual memory.
There's a reason. So, the first thing I would tell anybody that's trying to install this stuff.
I don't have those profiles.
Take whatever. Whatever distribution you've got.
Take the installer, ignore. The default layout, because it's all going to be partitioned.
And what's going to happen with partitions. Is VARTEMP or VAR log?
Is going to overgrow the system and freeze it. Because you can't manage them, because they're not on different volumes.
Ah, uh-huh.
Can you hold that till… into your presentation? Because it… right now, that doesn't make much sense to me.
Well, I… the point is, if someone's… if someone's making the transition.
Repeat that later.
Someone… Steve S…
Uh, do them a big favor, and… have them read… the, um… the Linux and Unix sysadmin Handbook.
Because they show you how to do it. I mean, it makes life a lot easier.
I might mention Steve S. In the chat is asking a link for the PDF version in the chat would be helpful for him.
Thank you, okay. And uh… Uh, Steven, if we can impose on you, if you could type into the chat, uh.
Uh, the name of the, uh… It's been a cable.
Let me find a link to it. Let me find the chat. Oops.
We're gonna jump… jump ahead and get done with this introductory stuff.
Um…
Hang on, let me… What did I… oh, there it is, okay.
Uh, so… Okay, the slide up there now is the slide for this group. Our website, address, all that stuff, so that's how you.
There's a project out there to helping people find Linux support.
And I've tried to… I'm working with it now to try to get this.
Entries show up properly for the new Linux user group. And I've been going back and forth with the developers out there.
Right now, it's… it's… pretty much geared toward.
Of people at… provide commercial support or particular events.
That's neat.
And it's not really geared very well to online events, and uh… I got to work with that yet, but I wanted to point out that.
Is that H… gonna be, hopefully, uh, very important in the next month or two.
Um, that's all I wanted to say about this slide, so… Coming up.
Next slide. Gary, go ahead, please.
Trevor is watching it.
Okay. Okay, uh, privacy and security. As I mentioned earlier, uh, not only are we recording this, and we're going to put it in our archive.
But even when we say our recording is done, if we're still up and running, anyone can be recording online.
Um, and uh… Even if we don't put something in our archive, we're editing it out. If there was a terrible oops.
Somebody else could have it, so just be aware, uh, that's… Not just with us, but with any meeting you go to. It could be being, uh.
Recorded. Next slide. Presentation archive, as I said. Our stuff is in the presentation archive.
Very useful. A lot of… Good material has been kept in there, so… Um, you're welcome to it, and uh…
Yeah, let us know if you enjoy it. Next slide.
Our calendar, as I already mentioned, we, uh. Uh, maintain a calendar, not just in St. Louis events anymore, but now just any events that we know of that you can access online.
Um, we'll put it out there on our calendar if it's.
Something about open source, or open standards technology. Uh, especially if it's Linux or Unix-oriented, so…
Mailing lists, uh… Uh, as I said, our discuss list.
It's basically like being here at a meeting. You can post any comment or question and see who, uh.
Uh, who has an answer or response for you. The announce list is more of a limited, uh.
Amount of things that are put out there, it's basically edited.
And, uh, basically, it just usually just mentions our meetings each month. There's something… if we know there's something that a lot of people might be interested in, we'll put it in and out. So, but it's… much more limited traffic than discussed.
Steering committee, that's where you see the sausage being made. Anyone's welcome to help try and run this group.
And, uh, that's where we do it. Sysadmin, that's the volunteers who keep our particular computers running, and uh… the people who are on that.
Group mailing list, actually. Do the work to keep the machines going.
Next slide. Uh, our sponsored meetings. Obviously, you're here for the St. Louis LUG meeting.
Uh, as we said earlier, there's also a general monthly meeting.
Which, uh, can be on any open standards topic. Unix in general.
Um, the new user group. And, uh, the Slack meeting, uh, I already said what those are about.
Next slide. Call for help. I already mentioned that if anybody wants to ask a question now up front, you can do it before the presentation.
And if I don't hear anything, we're going to go right on.
And, uh, so our presenter tonight, uh, he's gonna… tell us all about?
Logical volume manager, but first… Last chance. Anybody got any questions before we start?
Okay, Steven Lumbark is the principal at Workhorse Consulting. Uh, Steven, uh, used to live here in St. Louis.
And, uh, was a valued member of our group back in those days. He still was a valued member of our group, because now our group meets so much online.
Especially when desperate.
Hey, that's how you know somebody is valued, is when… When things are desperate, you can turn to that person and say, hey, and he always comes through with some good stuff.
So, as I say, Steven's been, uh. Been with us for many, many years, and we really appreciate it when.
He can, uh, speak to us, so… With that, uh, I will turn it over to Steven.
People see my screen.
Not yet.
Okay, because I shared it. Chair… desktop one, click.
Share real quick. There we go. Now I should be able to put this in slideshow mode.
There you go.
Abra Kadabra! Hey, it worked! So, does anyone here actually use LVM?
This is Stan. I use it on my laptop by, uh… Encrypting the laptop, it… goes through LUX, and it… it mounts the LVM.
Yeah.
But, uh, I haven't… I find it kind of complicated. I'm not… I'm used to the more standard.
Old kind of mounting. And, uh.
Anyone else? Okay.
I deal with it. But it's…
I've made extensive use of LVM over the years. But have mostly migrated to ZFS.
Um… Yeah, the…
I still prefer LVM, but… The main thing is, there's… part of it is, you know, do one thing and do it well.
So, LVM is nice. For saving you from having to deal in the physical hardware.
And it's also nice, by saving you from having to pre-allocate all of your disk space. So.
Um. People ask, you know, how is this logical?
And the name comes from the fact that it isn't physical.
Does everyone here understand virtual memory?
Yeah? Well, at least I… well, yeah, at least I used to until, like.
Not all said yes at once.
Well.
I kind of took a long break, but… from my, uh, from my memory.
It's not… it's different from the addresses on the physical RAM.
It's something that is set up virtually, like, not actually tangible.
And it's formatted differently.
Well, the… what… The point of virtual memory is that you can actually have more memory allocated than you physically have in the machine.
Because your program works in a logical pages. Which are mapped into memory as needed.
Basically… LVM is doing the same thing with a disk drive.
So, what you do is you take. A partition.
Or a physical disk. And you can say, I'm going to allocate the blocks.
To logical devices. Without having to fiddle with the partition description on the disk itself.
So, if you look at it, you get into all the little nooks and crannies of it, yeah, it looks like there's a lot of stuff.
But you'll find… all of these go… by themselves, pretty simply.
You have to look at it one piece at a time.
So, the reason for LVM goes back into the 1970s. And, you know, large drives were 10 megabytes. Anyone ever seen a full-height 10 meg drive?
Yeah.
I use them as bookends. They're too small for boat anchors, and they're too big for doorstops.
Yeah.
Mine's 2TB. Not 2TB, 2GB.
2gb. Well, I know, I've got a bunch of old Deck 10 megabyte.
Scsi grinds. And… but, like, they're, you know, a little bit bigger than a…
A textbook. At the end of the year, they're 4 inches deep.
Anyway, the problem was, you wanted to have disks. Uh, the discs were 10 meg, which wasn't necessarily enough to boot everything.
Booty from Tape took too long. The other problem is, if you needed swap space.
You sometimes couldn't take a whole disc for swap. You had to be able to allocate the space.
Uh, tar overlays were a pain in the ass. Re70 packs require physically removing the device to use it.
This is all archaeology to most people today, but. This was the problem you faced. How do you take these drives.
And manage them effectively. When what you needed was bigger than what was there.
So, the original thing was partitions. You would take a drive, and allocate pieces of it in cylinders.
For swap, booting, the operating system. On Unix, those each became their own mount points.
And every time you needed a new mount point, you got a new partition.
So you had to add drives, you partition it, you mount it.
One problem was, uh, with biostrives that are used most of the space with Linux.
The… you can only get 4 partitions. So, once you've… cut things up four ways.
Now, if you want to change something, you're stuck editing the physical locations on disk.
Uh, anyone here use Part-Ed? And if you ever make a mistake with Part Ed and fry your file systems?
You're…
Yeah?
Yes, with… Oh, yeah, that's fun.
Oh, yeah. Isn't that fun when you get to say, oh, goodie, now I get to restore all of it?
Don't. So, what you used to do, partition it. And then…
With Parted, you'd have to change things around without frying them.
And then every time you got a new disc, you had to decide what partitions.
To use for what? And you… you have to… the way partitions work, you effectively end up having to pre-allocate.
The entire disk for something. Another bad thing about partitions that make them hard to manage is they must be contiguous.
You can't… you can't take something and just extend it. So, let's say something got really, really huge, and again, in the 70s.
80s, 20 meg was a lot of data. How do you deal with something that big? If you have a file that large?
So, what happened was, there was an abstraction. Instead of having iNodes.
Which were attached to physical devices. Unix with BSD added V nodes.
Which were actually originally for NFS. So, a virtual node.
The idea was there, instead of being attached to a physical device.
You could attach it to an abstraction like NFS. Or a logical drawing.
So, two companies back then, uh, Veritas and Hewlett-Packard, developed their own ways to do this.
Veritas's way was a lot more complicated. Thankfully, Linux took HP. If you can find an old HP manual.
On how LVM works. Because LVM was HP's real selling point for HP UX.
They documented really well, and everything is… is… identical to the way it's done on Linux now.
So… Much like you would use FDIS.
Pvcreate. Takes physical storage.
And usually, the suggestion is use a partition. You can use a whole disc.
But it… it makes it easier to screw yourself up over time.
But you'll take a whole disk or a partition, and you'll say.
I want that labeled to be used. With LVM.
Vgcreate allows you to pool multiple disk. Drives multiple partitions.
Into a single unit. For reallocation. And this was how you could take.
Multiple drives. Or today, with Linux, you can take drives and mirror them.
You can do all kinds of cute things. And you can create a volume group.
That says, take these physical volumes. And group them together, take all of their storage.
And call it one unit. Uh, you can think of this a lot of the same way as Linux takes.
Multiple SIMs in your memory. And creates one pool of memory out of each of the physical SIMs.
You don't say, gee, I want to allocate a piece of.
You know, that ship or that ship. You just have… umpteen gigabytes or terabytes today of memory.
And it's one unit of memory. It's allocated in a virtual space.
Zione. And then LVCreate is how you take those blocks and allocate them.
Into the physical volumes. Or… or into pools, there's a whole bunch of things you can do with them. Now that you've created that pool, you can allocate it in a lot of different ways.
Did this make sense the way I said it?
Okay, at least one… everyone with a heartbeat say, hi, Steve!
Yeah.
Hi, Steve. Hi, Steve.
Hi, Steve!
Okay, okay. Okay, so we got a few people that are at least alive.
Now…
Hello, Steve.
Hi, Steve.
Now, one of the great things is Grub 2 speaks LDM.
You no longer need a boot partition. You can set up your GROV config.
To hold the grub config and do all the booting. From a slash boot.
That is mounted in your root volume. You could do that with a partition also, but LVM has some advantages.
I'm sure that got garbled. You said you can do that in something, also?
Yeah, oh, wait a minute. Hang on a minute. I updated this…
Oh, guys, I'm sorry. Um…
Got the wrong slide deck?
No. When I first gave this, everyone wasn't stuck using UAFI.
So I'd, uh… So today, you need 3 partitions.
With Linux, because you're using Grub. Your UEFI partition can be a few kilobytes.
I use 4MB because it's a cylinder. Uh, you need primary swap if you want to hibernate.
And then you can put everything else in one big partition.
So, SDA123… NVMe P123.
Uh, swap traditionally was twice RAM. You know, in today's machines.
If… if you've got a terabyte of memory. You don't need twice that.
And you can just allocate LVM for everything else. So, with UFI.
Again, the primary use for swap. Is hibernating today. A lot of systems, you've got enough memory on them that you'll never… you'll never use it for swapping.
So, if you're setting up the system, and this is an example from Gen 2, where you do it manually.
You can do the same thing with OpenSuse. When you go into the installer.
Uh, you… you skip. Their default.
Partitioning scheme, and just choose your own. And then once you get in, uh…
If you're installing Red Hat. Same thing, you go into the partition manager.
Ignore their defaults. You can create a trivial.
Space for UEFI, because all grub stores is one hook in there that references something else.
And then… you can have… the remainder of your space.
Just be the LVM. What you then do is create a volume.
Allocate it back out, let's say day one, you just have root and var and var temp.
Then you can make… do the MakeFS for it and mount them.
So, looking at it, if I ran FDISC, uh, again, let's say I'm using a SCSI.
I might have 8 mem… 8 meg for the grub hook. A lot of systems, they won't let you create a UAFI.
Smaller than 128 or 512 meg. Because they think you're going to store windows in it.
Uh, that's one other advantage to Gentu. You take SDA2, let's say we make it our memory.
Nobody swaps anymore, there's too much memory on the machines. And then you can allocate everything else to LVM. You don't need a boot volume.
Uh, an example from my notebook here. Is I'm using NVMEs.
So, NVMe 0 and 1. Has… is my… the first device.
On the zeroth controller. And that… you can…
I've got P1, P2, and P3 on it. For UEFI, Swap, and OVM.
And again, the swap is equal to the… the amount of memory I have in it. Notebooks with 32GB. If I told you that 10 years ago, what would you have said?
So, I do a PVCreate, I get a little thing like this. Created successfully!
Yeah.
Not very hard. Most of what that does. Is it puts a UUID.
Onto the volume. It just labels it uniquely. For LDM to keep track of it.
I do a VG Create. Uh, people that came from… HP will always call them VG…
Number number. That's just the way HP did it. A lot of people will have VG0.
Be… Red Hat people call it VG Root.
One big advantage to using logical volumes. As opposed to partitions.
If you've got 6 drives installed, what is on NVMe2P3?
Well, if you have a logical volume called var, where do you think it's mounted?
Or var-temp, or scratch, or home. You can name these intelligently.
So you can tell what's on them. So if I create, let's just say, day one, I create a root volume, a var, and a var temp.
One of the nice things about doing it this way is.
A classic problem you have. Will be that VAR.
Or VAR log. Will get eaten by VAR temp.
If you just have one partition for everything. You're… you're… you're fighting for space.
With HOME and VAR and LOG and Temp. And any one of them can outgrow.
The whole file system and sees up your machine. If you use logical volumes, it becomes very easy to allocate the space to individual things.
And to not allocate all of it. When you first install the system.
Because the logical volumes can be extended later, and I show an example of how that works.
I can say, you know what, let me allocate. You know, a third of my space.
And leave the rest of it sitting there. When I need something, I can grow into it. I can add a new logical volume on the fly, whatever I want to.
Can people… does this make sense, the way it looks here?
Can you see what I'm doing?
I'm starting to understand it, uh…
Okay, well, again, I'm creating 3 logical volumes. Once you'd be mounted on the roof.
Right.
One is going to be mounted on VAR, and one's going to be mounted on VAR 10.
On my own systems, I'll also create. Uh, var log.
And home, and what are they called? Scratch, the… I just use because it's not managed by the system, but it's kind of like VARTemp, but I don't have to back it up.
Another neat thing about this, talking about backups. Is… I know that I'm not going to back up the contents of our template.
So, I can… I can… by creating smaller volumes for these, I can back them up in parallel.
I can use different tools. To snapshot the individual volume.
Or I can do a find minus XDEV. And back up only things that matter.
And this makes it easier to segregate things you don't back up.
So, for example, Scratch. Is for things that I check out from somewhere else.
So all of the… the… the checked-out Git repos, all the stuff I've downloaded from a site so I can build an executable.
That I'll… if Scratch blows off, I say fiddlesticks, and I just… reloading. I grab it from someplace.
Uh, I use XFS… There are a number of nice things about it with LVM I can show people in a…
If we have time. One of the great things about LVM is I can create small logical volumes on one device.
To use as external logs for XFS to improve the performance, because I can… I can divide the I.O. Into different sets of lanes.
But here, I just… I'm creating a… I'm creating a set of typos. I'm an idiot. I apologize.
I can't do that.
Before you fix your typos, I think…
I was hacking… well, I was hacking this…
As you're fixing your typos, just remember that last section you just mentioned.
I'd like you to go over that again. I didn't…
Yeah, well, it doesn't fit. Anyway, the point is. You can create… I'm not sure if an underscore. Anyway, you can create names for these.
That makes sense. Var, var dash temp. Um… Then you mount them.
Um… God.
And… that's why you don't edit your slides 5 minutes before you give the talk, so… what I can say is.
I can mount all of these things. And the logical volume names give you an intelligent idea about what the mount points are.
I suppose the partitions. If you look at a partition, you've got no bloody idea where it's supposed to go.
If I removed, on most large systems, if you blow off Etsy FS tab.
People are going to spend hours figuring out where everything belongs.
With logical volumes, you pretty much know. Now, if you look at the Etsy FS tabs on a lot of systems.
People will use disk labels, they'll use UUIDs. Uh, you've got the whole issue that the SCSIs aren't guaranteed to come up in the same order.
So you have… the device paths don't work, and now you're mounting a UUID, and you look at the UUID, can you really tell if it's correct by looking at it?
No. Lvm assigns UUIDs to everything for you.
So the physical volume devs something, or the volume group VG00, or the logical volume.
Uh, VG00 root. They have UUIDs associated with them. They're unique.
So at that point, VG scanned. Will…
Go down the system, it will find all the storage volumes.
It will find all of their physical, logical groups. All the logical volumes.
And it will assemble them so that LVM. Can manage them properly. So now.
This is what an FS tab looks like. So I can… I can mount VG00 root on Mount Root, and DevVG00 root is repeatable.
Because it's all based on UIDs. This is an example where I used MD Adam.
To create a huge volume out of. Mirroring several things.
And then, once that's running, I scan it, I find the physical volume.
And then I can activate all the volumes and mount them.
So, this is actually a few lines from my own init script.
I've been using the same init on my server for about 20 years.
I haven't changed it. Every time I rebuild the kernel, I don't need to rerun grub and wipe out my grub config.
Because this is the same thing I do every time. I don't have to care.
Because VG00 root is always stable. This is also great.
If you have to do a system upgrade. Because I can create a devVG00 new-root.
I can do my full install and do a new root volume.
I can boot it, I can see if I'm happy with it.
And then I can hack FCFS tab and replace root with new root.
Bingo, I'm done. So, that kind of flexibility is what you get from LVM.
So, let's just say… my root volume group. Remember, I created this at 8GB.
And one day I say, you know what? It isn't big enough anymore.
I can do an LV extend. Now, the beautiful thing about that.
Is there's nothing in the universe that tells me VG00 root.
Has to be contiguous on the disk drive. Partitions do, that's why you have to fiddle with part ed.
I don't have to shrink any other logical volumes in order to grow this one, assuming I've got free space.
So day one, I allocated a quarter of my disk space and said, you know, when it happens, it happens.
I can do XFS GrowFS. And I don't have to unmount any of this.
You can actually have a daemon running, and people have written them, it's a… It's a good exercise with a.
A shell script, you can run something every minute. That there's a DF minus M.
And it says, ooh, is anything over 90%? Oh, wow, this one's at, you know.
92. Extended by 10%. Done. And the file systems just grow as needed.
Does this make sense to people?
Yeah. Yeah.
The thing that matters about it is right here. I can grow this with… oops…
Without it being contiguous. Just like virtual memory.
I can put the pages of your job anywhere in physical memory that works right now.
They don't have to be contiguous. You could have a data structure that extends over 8 pages. They may be all scattered all over physical memory.
But there are 8 contiguous pages in your process on units.
The same is true here. I've just said that these blocks.
Are now part of VG00 root. And there's… there's a fun story that I'll tell you later about that.
But again, I can extend all of this without unmounting anything.
I can add a new disk drive to the system. Let's say I add SDB here.
And… Uh, 8E is the type, if you're using old BIOS, 44 is the type if you're using the, uh.
The G-types, the SG types. For the part… for the partition types.
But I can do a PVCreate on it, I can extend VG00 onto it.
And I can extend route. And if 5 minutes ago, SDA was full, I'd used all the space on it.
Or SDA1? I can now extend root onto another disk drive.
And grow onto it. So I can add a new device on the fly.
Grow onto it, extend the file system, and keep going. That can be a really bad idea for your root volume, but.
This shows you how the syntax works. And again, if I stick a new drive in.
Same as before, I do a PVCreate. I can, instead of VGCreate, I extend onto it.
Which says, this story in SDB1 is now. Part of the pool in VG00.
And now I can extend the logical volumes onto it. Does this make sense?
Yes.
Kind of, sort of, yeah.
Yeah, sure, sure.
Okay. Now, again, XFS allows me to grow a mounted file systems, so I don't have to… I don't have to unmount anything.
So, again, going, you know, let's say you've scrounged 10 disk drives.
You know, which might give you, wow, 100 megast storage. Or in today's world, you might have… you know, a rack with a whole bunch.
Of SSDs stuck into it. Or you might have a whole bunch of NVMEs on one machine, because you're trying to get a petabyte of storage for doing something.
You could end up with one very, very large volume group.
And the problem there is these become fragile, because if one of your disks fails.
The entire VG will fry. This ha- um…
So, the fix here… Overly large VGs can become fragile. You want to break things into multiple volume groups.
And I can show an example of how I've done that on the server here. This happened to me once when I was working for HP. I took.
One really easy question that a field tech had. He was trying to do some work for a customer and asked her really simple question, so I figured, oh, that's great!
It's a quarter to five. Why don't I answer that one, and I can get out of here?
And it turned out that some guy had… added…
Something like 6 or 8 new disk drives to the machine.
But he didn't understand how to create a volume group. And he'd extended… All of his logical volumes, but he'd been very careful to extend the volumes.
On the individual discs. I know one of these drives went bad.
And so we told them, look, you know, you really should be using multiple volume groups.
Well, the cute thing is. If you wipe out a volume group and create a new volume group.
And then reallocate the logical volumes in it to the same physical blocks.
Hpux doesn't care, it'll just use them. So we ended up spending two and a half days.
Getting a complete list of all the… the list of the blocks, and rechecking it, and re-rechecking it.
And, you know, 20-something hours later, we had broken his… this huge thing that spanned 8 drives.
Into 3 new logical volumes that want… the route was only one disk.
And then the others were smaller groups of drives, that if one thing failed, it wouldn't bring down the entire system.
So you can do that kind of surgery with these things.
Um, I wouldn't recommend it, but… just the fact that we can save this guy's neck.
Was because we'd been… he'd been using LVM. And you've been careful about it.
And we can remap the whole thing. So, again, let's say I plug a new drive in.
I can create SDC… It's PV created.
Now I create VG01. And…
I can now create home. I say, you know what, home's getting too big, people… all this homebrew garbage is putting too much junk in people's home directories.
Just create a new… just give me new volume for that.
I can create this, bang. And again, it's called VG01HOME.
If anyone looked at this, would you make a pretty good guess as to where it's going to be mounted?
On somebody else's computer. That's their home, not mine.
Well. Yeah, well, it's gonna be mathered on a home.
Sdc1?
Um…
Now, one thing about backups. One of the classic problems is separating out files that you don't want to back up.
So people have got all this crap in their home directories that it doesn't need to be backed up.
Um, you can then take something… let's say I create a amount point called slash scratch.
And that's only for things that are checked out from somewhere else, that if it gets blown off, you can just.
Recheck it out. I can create a directory there for each project, for each person, anyone that wants to build stuff, whatever.
And they can SIM link that. Or I can bind-mount it.
Wherever I need pieces of it. Um, I can then…
Do a find minus XDEV, which only finds on the current device.
Of slash home. And it won't find all the stuff that's in Scratch, because it won't skip devices.
So, using logical volumes, I can create things. It's easier to divide the file system into smaller pieces.
By functionality. Where, if you use partitions.
You end up with really big things. And now you're stuck with, you know.
All kinds of different schemes for separating things that get backed and don't get backed, and… gripping them out of a path, or SIM-linking them, or whatever.
With LVM, I can either mount things wherever I want them, I can do a mount minus bind.
To put things onto… to LVM. And XDev avoids.
Backing up the parts I don't want. Does this make sense?
Okay, at least somebody got it. Now, with more smaller things, yes, I'm gonna… my Etsy FS tab grows. Oops.
My screen just went blank.
Yep, ours.
What the heck?
I think I did that. I clicked on this annotation button.
Thank you very much. Let me stop sharing… hang on, let me stop sharing.
I'm going to share again. Aggregate it, you know?
I just clicked on the annotation button, and the whole thing went black. I had no idea it was affecting you.
It's back. Slide your back.
Oh, guess what? Anywhere. But now, one of the things I can do.
Let's say I have more… I can take LVM. One of the classic things you've got.
Things like the root volume. And the stable portions of VAR, like VAR Live.
And home, you probably want those mirrored. At the same time, you might have other things like Scratch.
That you'd rather have striped. You want the space, it's… it doesn't matter.
You also might have… different file systems. You might have a reason to use EXT4 for something because it has to be used.
Or you're testing something with a new file system. With LVM, I… let's say I'm testing out a new file system. I want to check the performance on it.
I can create a logical volume. Test a new file system.
And remove it! If I were using partitions, I didn't have… I wouldn't have that flexibility. I could create a new mount point.
I can create a directory, excuse me. But it's hard to create new things for mounting.
Where with… with LVM, because it's file system agnostic. I can have any mixture of XFS, EXT4.
Nfs, ZFS, whatever. If someone insists that they need better FS for something.
Create a logical volume, format it. All yours mounted there. Done.
So this makes it much easier to have special purpose allocations or temporary ones.
Because I can create the volume trivially and drop it later when I'm done with it.
A really nice division of labor. Is…
I can make a physical volume. From an NBAM… an MD Adam mirror.
If I'm striping, it's just as easy to do that with OVM itself.
But MD Adam is nice. People, you know, people like it for using… for mirroring.
I can then do a PVCreate. Of the MD Adam mirror device.
And then create a volume group of that. And then I have, you know, I'll call it VG-mirrored or something.
And all the stuff that has to be mirrored can go out of that.
And it… the main advantage to that is it saves a little bit of syntax with LVM, because you don't have to compute the mirrors and tell it the device names and everything.
All that mirroring gets taken one level deeper. And you don't have to think about it again.
But then, if you take a rate volume, and you use MD Atom.
To make a RAID 5 out of a bunch of… you know, terabyte NVMEs.
Lvm makes it much easier to divide that space up and allocate it to people.
If you've got 5 groups that are trying to use the space, and you have one mount point, people will kill each other because they use each other, you know, Azog out of space, no, he's hog in the space, all that goes away.
Because again, allocate a different logical volume. To each of them, and everyone gets their space.
You can even over-allocate the space by creating a pool. And create what are called thin volumes. I'll show you that in a minute.
A quick question? Did I… Visit, did you define MD Adam?
Oh, MD Adam is the… the Linux utility.
For multiple disks, multiple disk administration. It's how you normally mirror things on Linux.
God damn it. Zoom sucks.
Um, so I'd normally RAID… RAID was traditionally done. Using the MD Atom utility.
To combine partitions. Into a single, larger device.
And then OVM is… makes it easy to break it down.
You could also do the mirroring in LVM. If I do an M minus M2.
This is a gotcha. If you want to make a mirror.
You can do an LVCreate. Until it… let's say you want a minus M… if you do minus M1, you'll add one mirror to the data volume.
And then you can actually specify which physical volumes to put it on.
It works, but it's a little more syntax. That's the advantage to doing the lower level RAID.
The rating is done, and then your LV atom, your… LV create syntax doesn't have to involve all the rest of this.
One of the advantages of doing it at the logical volume level is you can have.
Your volume group of four discs on your system, and one volume.
In the same volume group can use one type of RAID.
A different volume can use a different type of grade, or J-bond.
Yes. Yeah. It's a trade-off whether you really need that flexibility or not.
Yep.
Um, when we get done. I'm… I can show you on my server here, I've got several of these.
That are rated. As mirrors for things like home and my photography stuff.
And several that are rated as striped. For the scratch space.
So I can get a lot more space, and it's faster.
Another big advantage to using LVM to do your mirroring. Is the blocks for the mirror don't need to be contiguous.
That used to be something of a problem because of seek overhead on spinning drives.
But since SSDs that went away, and with NVMe, it's a non-issue.
So, one key thing is, I can create a mirror today.
And if I create it on two physical volumes. I can come back next week and LV extend it.
And assuming there's space, I will grow the mirror. And I just do L… I don't have to care that it's mirrored when I do the LV Extend.
It'll extend it on the devices that I ask it to.
And it'll extend the way I want it. Done.
Um, so again, I… MD Adam is nice for RAID.
Because it saves you a little bit of syntax when you create the logical volumes.
Now, one other thing you can do with LVM if you're doing rating.
Oops. You… you can set a RAID page size, which is the stride.
So, let's say, for efficiency's sake, on NVMEs. You've got larger than 4K.
As your page size. You might use 8K or 16K. Um, you can come back here.
With LV, and you can tell it. For… for the RAID 5, let's say.
I want to have 16 pages along each of the drives.
And then the logical volume block size matches the lower level MD atom size.
You can do the same thing if you make your RAID with LVM.
And at that point, you're doing it to tune the I.O. For the different devices.
Stride has nothing to do with mirrors, because all you're doing is… is… reading the same thing from each drive.
It may have an advantage, depending on your physical storage. Some types of physical storage prefer to be accessed in larger chunks.
Um, it's… if you've got… NVME with a lot of lanes.
On PCIe. You might find that it's faster to move a big chunk of disk at once.
So you might, instead of a 4K, you might say 8.
Or 16K. And it just… it saves you the overhead of latching the bus.
Because the data moves so quickly. Did that make any sense?
Yeah, it's kind of interesting, yeah.
So. You can create monitored OVM. When you do the LVM create.
You can… you can flag it as monitored. And this will track it for I.O. Errors.
You know, you can find out at 3 in the morning that there was a bad block right on one of your LVMs.
Today, with… SSDs and an NVMEs.
The number of… the rate of I.O. Errors is going to be so trivially small that you'll never see it.
But this does help if you've got a big cert… like a, you know, a central mail server or something, where if it has a… Uh, an OVM issue.
You're gonna fry people? Where you're gonna get fried in the morning, so you can have something that notifies you.
Um, so, stable file systems. People ask about backups. You know.
You can kick everyone off the file systems, let's say, every day at 3 AM.
And then watch the backup run and go back to sleep, right?
Everyone likes being awake at 3 in the morning watching tape drives, don't you?
No. Anyway… Uh, a snapshot.
Is not a hot copy. It does not do what people think it does.
What you create is a pool. In LVM.
Of copy-on-write blocks. So what you're gonna do is say. I want to back up the database right now.
And so, the database is going to be active while I do the backup.
And I might write… Oh, I don't know, uh.
10 meg of data at worst. So, I can create a snapshot pool.
That has 10 megabytes. In today's world, make it 20, it's not that big.
What happens is, while I've frozen the file system. Any rights that would have gone to my original logical volume, the one that's frozen for the snapshot.
Will instead be written. To a spare in the snapshot pool.
So the snapshot pool. Is a list of changes.
That would have happened to my logical volume. If it hadn't been stabilized.
Do that make sense? Anybody?
The brain is capturing up. The brain is capturing up. No, that's how a snapshot works.
Well, so you were asking about, let's say you're… you're asking about how to mount… let's say we're trying to back up home, and people use it 24 hours.
And let's say I've looked at it, and I've never seen a day where we had more than.
20, you know, 10GB of data changed. During the backup.
Make it 16, round up. What I will do is create a snapshot pool that's 16GB.
And I will freeze. The LVM that's mounted on slash home.
Now, anyone changes something. All that happens is their change gets written into the snapshot pool.
The original LVM DevVG00 Home. Doesn't change.
So this is a look-aside list that says, okay. This change. Well, people can keep using it. They don't know the underlying storage is frozen.
Because there's the pool. Add space that's dynamic.
To the logical volume. So that the original logical volume space can be stable.
Did it make more sense when I said it that way?
Yeah, what I think you're failing to say is… once you have completed the backup, what do you need to do?
I'm getting there. When I turn off the snapshot.
Okay.
And I make the logical volume writable again. All the snapshot pool pages that changed.
During the snapshot window. Get written into the logical volume.
And then they're discarded. So what happens is…
I do a snapshot. I create a pool off in left field.
Of new pages that can be written to. Instead of going to the original logical volume, which is frozen.
All my changes go into this look-aside list. And then when I unfreeze it, the look-aside list, all those pages get written into.
My logical volume. And they're discarded. Um… you can think of it as similar.
To what you get. With a… or replaying a journal.
In a database. I can freeze the database, journal all the changes.
And then unfreeze the database and write the journals. So that's how you do a lot of database backups, is you freeze the database.
And you store all the changes that would have happened. You might have a mirrored copy of the database for that.
And then you replay all the changes to re-sync the copies.
The snapshots make it very easy to back up things like databases.
Or a home directory. Or, you know, if you've got a download directory, you're running ETL, and you have to have a place where you're writing today's files.
Everybody at 1 o'clock, snapshot it. Update it, replay it. The replay has some overhead.
Is you're gonna do a lot of writes to it very quickly.
But it usually, if you plan it carefully, you won't kill yourself.
And it makes it very easy to use something like rsync.
To back up a larger space, so you can take, uh, slash home.
You can rsync it onto another volume. And then…
Take that. Take the original mount point, make it rideable again.
And then take the rsync. And back it up, squash it, compress it, do whatever.
I do that here, and what happens then is, if something bad happens.
Happens to Etsy or Home or whatever. I can go into the backup volume, and I've got an RSync.
Of home, Etsy. Uh, VARLIBE.
Vardb, there are, you know, half a dozen directories that matter.
My repositories. And that's just, uh, a mirrored copy, not mirrored at the file system level, but mirrored with rsync.
And that will be overwritten tomorrow. Then what I do is I take the rsync.
I CPIO it, XZ it, and stuff that up as 3.
I know that that's how I keep the history of it.
But again, the snapshots allow you. If you have space to store the updates that would happen.
During the snapshot. You can freeze a file system and then unfreeze it.
Um, can I… try saying it a little differently that may… Um, think about it as you have.
Sure.
Slash home, mounted on the home logic volume. And you've got people compiling and doing things late into the night into the backup window.
So, you tell LVM. To make a snapshot space that's kind of like a…
Think of it as a list of the changes that are going to be made.
But they're made in the snapshot space, not the actual underlying.
Backing disk space. You do your backup.
Of the low-level disk space. Once the backup is complete.
You then tell the snapshot, okay. Commit those pending changes that people have made.
Well, one thing about that. Don't picture the snapshot as a log.
To the real disk space backend.
Because let's say somebody makes 14 changes to the same page on disk.
You're only gonna have the last one. So it's… all it is is a list of the disk pages that would have been up… that were updated.
Fair enough. I was trying to…
During the time the snapshot was in force.
Yeah, I was trying to simplify to make it easier to understand, but…
Yeah. But it's… a lot of people think of it as a log, and you're replaying the log, and it's not.
Uh, if… if you make. 45 changes to the same file in the same block.
You're only going to have one block to write in at the end of this.
How do you play the…
You don't replay it. You just wouldn't… you take a mount point, you snapshot the mount point, and when you do that.
You have another logical volume that you say, all the changes get written here.
How do you play those changes?
You don't play them. When you're done, you unsnapshot the logical volume, and LDM.
That's what he's asking.
Takes each… let's say… I've snapshotted… you know, the DevVG00 home.
And I put, you know, 20 meg of space over in left field.
And now, you try to write something, and it gets written to block number 202.
In the snapshot pool. Block number 202 is going to get labeled, oh.
Here is what this should look like. So it's gonna reach into the snapshot in volume, find what the contents of 202 look like.
Put those into the snapshot pool in a new fake block.
202. It's gonna write your changes to that block and go along with that.
So that happens when you mount it?
If you make 14 changes to block number 202, each of them is going to overwrite them in this shadow copy of block number 202.
But I'm… what I'm done with the snapshot.
I think what Stan's asking is, once you're…
It's gonna walk down the scratch space and say, oh. I see that this page was written to. Oh, this is your LVM's block 202.
The contents of what I see in the snapshot. Is going to be written into the original LVM in block number 202. It's gonna go on to the next one.
What's the command to do that last part?
Um… He'll, um… I can look it up.
I think that's what stands asking is, it sounds… Stan, it sounds like you get that you start…
The syntax for it, um, if you… I'll show you that in a minute.
Okay.
Because I have to get a browser for that. Um…
I can shoot… there's… if you look up the LVM Man pages.
Um. So I have a lot of databases, full directory, a disk cache, and a home dirs.
Um, so let's say I've got a backup. I've got varied Postgres.
And let's say there's a single logical volume for acute, and it… I want to back it up to gig per hour.
So, I'm gonna take about 30 minutes. So I want to keep 8GB free for my snapshots.
So I'm going to create… Um… thing called PGTAMP.
Of one gig. And that's twice the usual amount that I back up.
So I've… I'm going to create… Inside, I'm going to do an LVCreate of 1GB.
And this is a snapshot volume. Its name is PG Temp.
And I'm gonna… I'm gonna use space for it in DevVG01 Postgres.
That's what this is saying. So now, DevVG01 Postgres has reserved 1 gig of itself.
To make changes during the snapshot window.
If I try and write more than one gig, I'm gonna have a problem, because I won't have any place to store the changes.
When I'm… now, I can mount. Um…
Pete, um, PG Temp on Mount Backup. This volume represents.
The snapshotted version. Of my Postgres volume.
Now I can do a find. On this mount point.
Reading out of the virtual device I created that's frozen. I can do a find, I can do whatever.
This PG temp is the snapshot. At that point, when I'm done.
I can drop the volume. Oops.
I hit the wrong key, sorry. When I'm done with this.
I… you… I can take this… volume… And I unmount it.
And I'm… and dropping. And it becomes…
I should, yeah, I should have put that syntax in here. Um, we can look up the command later.
Is it LV remove the PG Temp device? Or temp elevate?
Yeah, you know, it's… you create it, you use it, and you discard it, is the easiest way to handle it.
Yeah. Um, one thing to be mindful of is that snapshot.
Um.
It is functionally… a crash consistency snapshot, as if somebody pulled the power out of the back of the system.
It on disk, it is a mounted file system, and I saw in your mount command you had no recover.
Yeah. Depending on how you do these, yeah.
So, different things behave better or worse. With…
There's… in the time I have to give the talk, I can't describe all of it.
Um, if you want to take a look at the Linux and Unix System Admin Handbook.
They have excellent examples of doing this. If you look on the net for Linux LVM snapshot, there are lots of examples that show you how to go from beginning to end.
But realize you can do this much of it easily. It's very scriptable, very easy to do.
And it… it makes it trivial to do stable backups. Um…
Another thing you can do, and this is… Really nice if you've got a lot of… a lot of virtual machines. You can create a thin volume.
This is something you absolutely cannot do with a partition. If you think about it, virtual memory.
On a computer can be larger than the physical memory of the device.
And what doesn't fit into memory, you swap. What I can then… another way to do the same thing with storage…
Is I can pre-allocate a large logical volume. That's effectively a pool of blocks.
That can be allocated on demand. That allows me to let logical volumes grow as needed, so I can pre-allocate.
More… I can pre-allocate a fixed size. I can create a bunch of logical volumes.
That… technically take.
More space than the total pool that's available to them. And so long as no two people.
Exceed the total while, at the same time. I don't have to manage each of the volumes.
This is great if you've got development environments where people are doing.
Huge database dumps, large compiles. The analysis of very large files.
When the… the… The logic, I can say, okay, you guys.
Your department has been willing to buy 1TB of storage. Each of you wants… 3 quarters of a terabyte, there are five of you.
Figure it out yourselves. Don't step on each other's toes. But any one of you that needs 3 quarters, you got it. Here are your volumes.
Two things are nobody can grow more than 3 quarters, because that's the logical volume I gave them.
And I create the files, the logical volume grows dynamically by slurping up.
Pool space. When I delete files, it gives the disk space back.
To the pool? And this can be a, like I said, a real lifesaver if you've got.
Things like backups, database dumps. If you're using KVM with scratch space, or QCal.
You can have something that grows and then shrinks. And the logical volumes will be dynamically maintained to the right size.
There's a slight performance overhead for this. I mean, you've got to allocate the blocks.
But it beats having to have. Five times the physical storage you actually need.
At the cost of people having to avoid stepping on each other's toes.
Um… So, thick.
Which is what we've been seeing so far, allocates all the logical blocks when you create the volume.
If you give it a minus minus thin. The logical volume.
Is given a maximum size. But it doesn't use any blocks yet.
When you do, let's say, a… an F… you do a Makeafest.xfs.
That'll write a certain small amount of data to the disk, so now my logical volume will be maybe 2 blocks in size.
Allocated 64GB. But I've used maybe 8 kilobytes to store the XFS feed.
So now I've used two pages of actual. Lvm space out of the pool.
If somebody then stores a… you know, a 32GB file.
The logical volume sucks up blocks from the pool. It uses them. Somebody unlinks the file, those blocks get put back in the pool.
So this saves you from having to grow and shrink these things dynamically. The great thing is, you can always extend a file system.
Shrinking a file system is hard. Because the file systems don't like to reorganize themselves to give up space.
So the great thing here is I can do a MakeFS, XFS, BetterFS, whatever.
And I can deallocate space from the logical volume. Uh, whenever the file system shrinks back down.
The XFS, or BetterFS, or ZFS, whatever file system lives on top of the logical volume, it has no idea those blocks are being given away.
So I unlink the file, the storage goes away, I create a new file, the storage grows.
And the file systems themselves are none the wiser.
How is the LV under the file system knowing that it can give.
Reclaim that space. Is it dependent on trim?
Uh, pretty much, yeah. Well, when you dealloc… when you unlink a file.
The… the file system is going to… basically, yeah, it's gonna deallocate those blocks.
The file system is going to deallocate it, but the logical volume under it still sees data there.
From the pilot.
Um… Yeah.
Unless the trim, and then the file system tells the line block device, you can take these back.
Right. There's a certain amount of synchronization, but it… the only thing I… The thing to take away from this is, in situations where you have storage that grows and shrinks.
You could have a thin volume. Accounts for that. And you can allocate more than your physical space to the sum of the logical volumes.
Um… And again, if you've got ArchivesSpace, ETL.
If you've got scratch face where people compile the, you know, the job from hell when they got all the temporary files.
This can be a real lifesaver in those environments, because you don't have to either create a logical volume each time and drop it.
Which requires super user privs. And you don't have to…
Pre-allocate a logical volume where you're. You have to have a lot more physical space than you actually use at any one time.
Uh, this was a lifesaver in the days of 10Meg drives, you know, or half-gig drives.
A university might have 6 RA81s. And allocate 5 times the space, people had to schedule what they did to avoid stepping on each other.
But it saved a lot of overhand management. Um, so the example I use here is if I got 30GB of free space and 3 users, and each of them.
Claims they need 48. Okay? I create a thin pool.
Uh, so I create a 30 gig thin pool called Scratch.
In VG00. Then, I do an LV Create, a 40GB of.
Thing, call it Thin 1. In Scratch. And then I do Thin 2 and Thin 3, let's say.
Now I can mount those. If I…
Lv Display shows that none of the space is used in each of these volumes.
And actually, these can be a real neat thing. If you use virtual machines.
Because you can allocate. The raw logical volume.
As the storage in KVM, and it just grows and shrinks automatically.
Because there's no file system above it, the KVM will talk to LVM and allocate and deallocate the spaces.
Um. When somebody uses the space, it'll be allocated.
Now I make a file system, I mount them. Df each shows as 40GB.
Lv Display, which is our VG display, which is how you look at the space used.
Will show that nothing's used yet, or some trivial amount for the overhead of the file system.
Everyone's happy. Um, if two people.
Try to use 15GB at the same time, they'll be happy.
If 3 people try to use 15GB, they'll step on each other.
Because you've allocated a dynamic. Pool, you have it pre-allocated the space they asked for. So again.
This has the caveat that people have to coordinate their use of it.
Um, oh, yeah, and if you do… I'll exceed the space.
Everyone will be killable blocked. So, normally.
If you fail on a disk-full situation.
You try and kill the process, and you can't kill it.
Because it's locked inside a box inside the kernel. I'm not going to go with all the details of that.
But things that have… an I.O. Pending.
Because there's an incomplete kernel call that can't be killed. One of the things that a thin pool does for you.
Is you can kill minus kill a process. That is waiting for the allocation.
Of thin pool blocks. Or it'll just continue if the blocks become available.
So this, actually, there's a use. For having a… a thin pool.
That uses up all the space. In the pool, I can have a thin logical volume that's, let's say, 32GB.
And the only reason for that would be. Let's say you've got a daily backup.
And let's say the data… the daily backup. Ask to do a dump and G-zip of a large database.
If they're dump and de-zip ever takes more than 32GB of the scratch space.
You're going to try and write a block, the operating system's gonna have nothing to write it to, and now you've got a problem, because you can't kill the process.
Because it's locked inside of a kernel call. If you allocated that out of a thin pool, I can do a kill minus kill on it.
To at least free up that space and restart my backup.
So again, the… The takeaway for this, without all of the details, is.
Thin pools allow you to create. A disk space that if something fills it.
You can still zap the process. I keep the system running. You'll obviously have to restart whatever it was.
But that beats having to reboot. To get the system back.
Do that make sense? Has anyone ever been in a situation where you filled the disc.
And you couldn't get rid of the process? That's what… this avoids that situation.
Yes, it makes sense, and yes, I've been there.
Um… now, what happens, let's just say you were doing your daily backups.
And… oh, crap, today somebody wrote something really big, and we just… we just… out-use the pool.
One thing I can do here. Is I can do an LV extend.
Onto a new disk drive. So, let's say…
You know, I've… this is desperate. We have to finish this back up.
Or no one can do anything. I run out, I get a thumb drive!
I stick a USB thumb drive in, and I VG extend.
Uh, the volume group onto that new device. This is in complete syntax.
I can then LV extend. My… my pool… onto that new device. It's gonna be slow, but it'll be there.
Now I've got free blocks. The rights continue. When I'm done, I can drop that volume and LV reduce off of it.
And get my thumb drive back. Again, logical volume management allows you to do things that are quick, one-time fixes that can't be done with partitions.
Um, this is the kind of thing that you would do on a high availability system.
You might actually have one hot spare drive. And one of the things you can do is, if one of your drives starts getting errors.
You can extend whatever logical volume sees the errors. Onto your hot spare.
Dupe the drive. Do it… you can do a P… you can copy the physical volume off the bad drive onto the new one.
Or if you're mirrored, I can take that one out of the mirror and re-mirror onto the new drive.
And then pull the old one out. And no one's the wiser. I don't have to unmount anything.
There are even commands in LVM. Where you can add a new drive, and then.
Tell it to remove. The original drive, and LVM will actually move the pieces.
Hot while things are mounted and being used. To the other drive, so you can take the bad drive out.
Yeah, it… and… you say, gee, you know, I'm not running a heavy-duty data center, why do I care about this?
But even on… if you're on your notebook. On an airplane, and you've got the talk that you're working on, and you're using an external thumb drive.
If you actually use LVM on the thumb drive, and the thumb drive starts flaking out.
You can plug a second thumb drive in and move everything onto it without having to… without losing any data.
I mean, I've used this in all kinds of insane situations.
You know, on long flights, on my home machine or whatever.
Uh, to save myself from having to. Backup and restore an entire physical volume.
F trim. Um, this allows… this is what you brought up earlier.
This takes all the files and… it communicates between the file system and LVM to know which pages are actually in use.
Leave it at that. So this is the magic that allows you to reclaim space from different files.
Um, depending on how heavily your file system is used and how big things are, and how often they grow and shrink.
You can cron this every hour or whatever. And it will keep things reasonable. You know, cronin every 8 hours.
And people will eventually get into a rhythm. Of knowing they can compile the program from hell.
You know, every so often. You can put this. Into a Jenkins job.
That does the build from hell, and just does an F trim on the volume on the way out.
And this can be really useful in an environment where you're compiling lots of things that are big.
Everybody knows you start, you use the pool up. And you do an F trim.
Instead of minus all, you do an F trim on the volume that you used on your mount point.
Go away. And that way, everyone uses what they need. By setting the logical volume size, I give everyone a maximum.
And everyone gives what they're not using when they're done. So this can be really helpful for people.
Um… Yeah, and these are… these are examples…
You know, if you have a dynamic environment. Um…
You can use a 3.0 kernel, which most of us have by now.
Uh, if you have… If I trim… you can have the file system, the kernel.
Will automatic. It will automatically do real-time trimming.
And basically, because the kernel knows about the deallocations. Uh, because it has to go through…
The, uh, the file system. To deallocate the pages.
The kernel communicates between the file system and the logical volume for you.
And there's a link to this. I'm not going to go into it now, but, you know, you look today with NVMEs.
This trimming is so bloody fast. But it's actually reasonable to use in environments where you.
Create huge numbers of files and then delete them all at once?
It'll… it'll just automatically clean up for you.
Oh, and the other note is, um, given this has been in the kernel since 3.0.
We can all agree this is not exactly bleeding edge. Um…
You can look in sysblock, SDA, whatever QDiscard max bytes. You can…
You can… you can find out whether discarding is… available on the file systems?
Um… you can do a mount with discard.
And that will enable it. Or you can…
Um… you… okay, yeah, you can put discard into the Etsy FS tab.
Or you can do it manually when you do the mount.
Um, this is a really bad idea on slow storage, like, uh, USBs, by the way.
It just… it puts too much overhead rewriting stuff too often.
You're better off doing it manually. For faster storage, like NVMEs, it works just fine.
Um…
And let's see, LVM, again, the main things you get out of LVM are senior mount points.
Which means that you can… you can allocate space. Put it under a mount point, you get more smaller allocations.
Where things don't fight each other. The thing most of us can picture is I can take.
Var log and VARTMP. I can put the two of those, I can set them to reasonable sides.
And if a log runs away, it won't crash my system.
Because var temp won't fill it. Var log will fill my logging daemon may backup.
But I won't… I will be able to fix it, because the entire system won't freeze.
So doing this by dividing up that space saves you a lot of reboots if something runs away.
Backups with snapshots. Dynamic Space Manglement.
Uh… bad pun. You can… with trim and thin pools.
You can dynamically allocate space, again, if you've got something like a… a fairly large file system that's got.
Qcals for KVM. When those grow or shrink.
Or when you drop them, if you drop a KVM. In a lot of environments where you create these things.
You might create a… a virtual machine tests something and discard the virtual machine, or you do a build on a virtual machine.
Because you want to specify the entire environment. You do the build on a virtual machine. When you're done, you toss the virtual machine.
It's QCal file goes away. You can return that space, and now you're…
Your volume management to account for the space of your virtual machines gets a lot simpler, because it's fully automated in the kernel.
That's it. Did most of this make sense to people?
A lot of it made sense to me. I'm gonna have to go over the recording.
A couple of times.
Um, so to give people a quick example, um… do something here.
That's a little bit bigger…
Yeah, double that if you can. Yeah, a lot bigger.
Oh, okay. Um, timeout!
Keep going.
How's that? Well, I gotta leave enough space to show what I'm showing, so this would be the best we can get.
That's almost readable.
Okay, so…
Um, so if you look here, I've got… DevVG00 root, you'll never guess what that is, var home opt.
Varlive DVD… is where I back things up.
Vg01 is mirrored space that isn't part of the operating system.
Uh, if I do a VG display.
Vg Display, um… wrap off the screen.
Okay, this is LVM2, it's read-writable. Um, it's…
Close to 2 terabytes. It's a 4MB disk allocation.
And I've allocated… you know, less than half of it.
Now, you see this volume group, UUID? If I do a… VG display minus V, it'll show me everything, but at the bottom, it'll show me the physical volumes.
So if I look at this one. Nvme6.
What I did is, well…
When I put this machine together. There are two NVMEs.
That are mirrored. To create the operating system.
So, they're duplicated. Uh, UEFI, boot.
And swap. Oops.
So, how do you see the relationship between those?
There we go. So… what I did is I created one of them, created the other.
And… the UEFI, the boot, and the swap. Were created with copies. I used X… just XFS dump and restore.
To copy things across. And the point to that is, if one of these goes out, the other one is a stable copy.
It also means I can update the system. Decide I don't like something, and reboot from the other one.
Or if I scramble the UAFI on one of them, I don't touch the other.
So each of the Macs is a… is a hot backup for the other.
But then I've got the LVM. In this NVMe 1P4.
So this is, um…
Oh, okay. So, so basically, the first three partitions there are duplicated on each drive.
Yeah, they're… they're copies. Um, so if I look at NVMA6 and NVMe7.
Okay.
I just got some stitches out of my hand, so my typing isn't all that great.
Oh, I want to… yeah, I won't.
Okay, what am I typing here? There we go.
Oh, that's right. Vg Display doesn't like multiple devices, I forgot.
Don't you want P4 on the end of that?
Nvme… No, I'm doing an FDIS, um… Oh.
Oh, goddammit.
Dash L, don't you want dash L to list?
Oh, that's why, because I should have used a minus L.
Okay, I'm an idiot.
It's late at night.
So, these are identical. Um, they're two of the same.
The same thing. So…
If I look at… The easiest way… VG00.
The point is, there are two of these drives. The physical volume is the P4.
So 6… oh, it's 6 and 7, not 4 and 6. So 6 and 7… Now, if I look at…
The individual logical volumes. The root here… You see this UUID?
That's pers… that's written into the disk. And when I do a VG scan, it says, oh.
I find all of this data is written on the disk drive.
I'm going to find a logical volume root, and this… Is it UUID?
So I don't need to put this mess. In my Etsy FS tab.
I can just put DevVG00 root. Because that refers to something stable.
That's unique across all the devices. And you'll see that the volume is 48GB.
Um, and there are two mirrored volumes.
Flag on the play. I've actually worked on systems that had.
Duplicated volume group names or logical volume names. For very specific reasons.
Yeah.
It should be, like, less than 1%. It's not… guaranteed that.
Yeah, you… Yeah, I could have VG001 root if I wanted a backup of the root volume.
Will be stable, but…
And I could have… I could do, you know. These names… the name of a logical volume has to be unique within the volume group.
Yes?
Um, but again, the point here is this is mirrored. These are all mirrored. Var is mirrored.
Opt is mirrored, home is mirrored. But I can extend any one of these when I feel like it.
So, um, I can do an LV display. Yeah, VG00 Home.
And I'll see this much, you know. It's 80 gig. I could extend that right now.
And all it would mean is home isn't… actually, I've extended this multiple times since I set the machine up.
And it's fine, it just means that it's not contiguous. 20 years ago, when all the discs spun in circles, yeah, that would be a performance issue.
Today, with SSDs and NVMe, it's not. Um…
I can also do a VG display at VG01.
And… oops. And you'll see this one is mirrored with MD Atom.
I did it that way… Because these things… the things that I create in VG01 are mirrored.
But I don't need the extra control. The operating system, home, var, root.
There are times when it isn't a bad idea to create a three-way mirror.
I wanted that… I wanted that flexibility with VG00. I didn't care about it in VG01.
So, this is a volume that's used, it's mounted as shared space.
In different, uh, KVM machines, so I can boot a KVM machine and copy a standard Etsy directory.
My backups…
Um…
Okay, Steve, hold on a second. You said you had two NVMEs in the system, right?
Uh, actually, I got 6.
Okay. Because I was wondering how you got… primary and two mirrors out of two disks.
I've got a bunch. So, uh, basically…
This is actually a really neat motherboard. Supermicro makes it, uh, for epics.
And one of the things you can do if you don't want to use SATA's.
You can disable the SATA controller, and it's got two NVMEs.
And all the data lines for the SATA controller get turned over.
To a pair of built-in NVMEs that are on the motherboard.
And have a priority lanes. So the operating system speaks to these NVMEs, and the swap is on it.
So they're built onto the motherboard, and then the cards have got the other ones on them.
What ends up happening is, um.
This stuff here, this is all my photography. Uh, these are my backups.
Oops. And I'm lucky I checked, because I filled up. Okay, I added… I added some new storage, so I can do…
So, VG Display has got 189GB free. Uh, right now, there's 180 gig in use in the logical volume.
So I can say LVExpand… minus L. If you use a capital L, you're extending it in bytes. If you use a lowercase l, you're extending it in pages.
I don't like dividing by 4 meg in my head, so I'm just gonna say… extend this to…
Um… 196… gigabytes. Gi means multiplied by 1024.
Yeah, VG01 backup. Okay. Now, this didn't make any difference.
But I can now do an XFS… GrowFS.
Yeah, VGs, you're, um… var backup.
Bingo!
One of the great aliases is FC-E-. You type r space at the beginning of the command, it'll redo that command for you. So now, I've got 16GB free.
So now when my housekeeping runs tonight, it'll find the file system isn't pull.
I could automate that, and I'm just too lazy to do it.
Um, what this does, actually, on each of the hosts has got its own directory.
And there's a daily rsync, so these directories, boot, etc, home, live, Optbar.
It does an RSync of those. With the same directory.
On the mounted file systems. Then it does a CPIO of that.
And XZZ. In…
Then that XZ gets pushed up. Um…
And it… it does an XZ into S3 into… a compressed volume.
So, that takes care of all of my, um…
So I've got these… that are stored here.
So I can go back 30 days, and I can say, oh, I want to recover Etsy from 30 days ago, and I can grab one of these, and it's there.
And then these get pushed up to S3, and they get rotated once a month.
I wipe out anything from the month before this. So it grows from 30 days to 60 days, and then shrinks back down.
So I can recover the system. Um, at least 30 days.
Uh, and sometimes up to 6 feet. And the amount of storage it takes on S3 is trivia. It costs me pennies a month.
But that, you just saw how much work it is, you know, if I decide, gee, you know, um…
Maybe I don't want that to be above. Maybe I should give it another 16.
Um… 96… so I can do, um…
16… 96 and 16 is 106, 112.
God, I wish I could type. Um…
Okay, that's how long it took to get myself the free space.
I noticed I didn't… I didn't dismount anything. There could have been 55 running jobs writing into that file system, and none of them would have cared about what I did.
They would have lost a microsecond. That's why I prefer LVM.
If I had had to use Part Ed. To hack this.
I couldn't have done it, because it would have been contiguous, I would have had to move 6 partitions out of the way, I would have had to juggle all kinds of stuff, the partitions would have had to be stable while I did it.
This, I don't care, because they don't have to be contiguous. I can just extend it and keep going.
That is predicated on there being free space in the volume group.
Well, that's why I started with VG Display. Vgs are 1, or there's, um…
And I've got 153. Another important thing about this. Is when I created these file systems, they were smaller.
I didn't… I didn't know how much space I was going to use for all of these.
So I didn't allocate most of the space.
And that's one of the things most… commercial distros do when they use LVM is they allocate 100% of the.
Which is idiotic, which is why I… the first thing I said is.
Yes.
Don't use. Any of the dis… the only distro that it makes sense.
To do the installation on is Gen 2, because you do it manually, and you do it right.
Every other one of them is botched. If you take the defaults. Suze is the one that makes it easiest to override.
Now, another thing is, if I do a VG display, a VG02.
Um, oops. This is on an MD that is striped.
Um, so what happens… is this takes up.
Two of my NVMEs. But it's striped in, I think it's 4 or 8 meg stripes across each one.
Um, so all of my scratch space. This is, um… there's a… there's a pool.
Um… Now, there are a bunch of these logical volumes.
Like this thing here, that is a logical volume. That it was used to create one KVM.
And when I'm done with testing tumbleweed in that particular, I'll just drop the logical volume.
Um, the one with the pool above it is used for an LVM pool.
Scratch is a… just a huge space. That is not backed up. Anything in there.
Has to be backed up somewhere else. But then I don't have to back it up.
Um, image in work, this is where I do my image editing.
The stuff I get off the camera goes into VG01, where it is mirrored.
The in-work stuff, if… if I… if one of those drives dies, I can always recreate my in-work process.
Uh, Postgres. In my case, I don't have any stable databases on here, so I'm better off striping it to use the space.
Um, and these, you know, these… DG02 Rescue was a test of a rescue system.
I was testing a KVM where I was testing out. You know, uh, DeepSeek.
This was a virtual… I was cross-compiling stuff. The point is, I can create these logical volumes whenever I want them, and drop them. It's trivial.
Another cute trick.
Um… you'll see these are all logical volumes. You see where it says EXT log on all of these?
I have one NVMe that is smaller than the others. It's perfectly fast, but it's not big.
And I didn't have anything to… I didn't want to buy something small to mirror it on.
So what I do… one of the things you can do with XFS.
Is you can create an external log. So that when you're writing.
To the XFS file system, the recovery law. Doesn't get written to the same device as the blocks. It just speeds things up, because you're… you've got fewer blocks being written.
So, all of the mount points for these. Um…
Log in. So… the scratch volume.
Has all this, you know, know our data, you know, quarter… Um, big log bee size.
This log dev says. All of the…
Pre… you know, the replay logs that allow the thing to recover from errors.
Are written onto the scratch. Ext log on VG03, which is on a different device, which uses different lanes.
And so that… that saves me… from relaxing lanes quite as often, so it gives me a throughput improvement.
If I lose that, I can recover the device by creating a new logical volume.
And going along with it. The trick is, if I used partitions.
I… if you look at these things, um…
You'll see that these things are… scratch. These things are 256 megabytes.
They're… in terms of file system, they are trivial in size.
In terms of logs, they're huge, because I got the whole NVMe for it.
But I can create all of these small devices. And I don't care. I can… I can allocate things in 4 meg and 256MB.
Chunks, and 512 made chunks, 64 meg. If you were doing this with partitions, you'd go crazy!
There would just be too many of them. But this thing, I've got a KVM QCAL2EXT log. I can tell what it is by looking.
If I told you that this was dev NVMe01P13, what is it?
You've got no idea. If somebody deletes your FS tab, you're toast.
This way, I can look at this and know that it's my… Inca's file system default EXT log. Finally, that's the XFS that's mounted for that.
So it makes it really easy to allocate whatever you need, when you need it.
I can grow it later if I want it to be bigger.
Although that requires fiddling with the file system. I can still do that with the XFS utilities. I could extend this and say, make the log buffer bigger.
If I start writing too much data. Um…
But that's, um…
So, you'll notice also here, I've got all of these names that mean something.
Then you'll notice this DM35. Um, what happens when the machine wakes up in the morning?
These are all symbolic links. To a… the real device.
So all of these DMs. Are different… virtual devices that were created.
Basically a boot time. Um, when… when I did that VG scan, it created all these.
There's also… you may see this a lot in distros.
You'll see Dev Mapper. Vg02 something to this. A lot of distros Red Hat does this.
Will mount dev mapper VG00-HomarMeta1 or something. And…
Vg's dev map or something. There's no reason for it, you're just as… you might as well just use dev slash BG00 whatever.
It's a lot easier to read in your FS tab. But these are something that's used by the device mapper.
Uh, to manage… the space for itself.
But note that these things need to exist. If they get.
If something screws up mounting the dev file system, and these don't get created properly.
Lvm's gonna be very upset with you. I can do a VG scan.
That's… well… So, that's about how long it takes to scan all the storage.
And it will locate all of the volume groups. It will make sure that all the logical volumes in the are created, it does all that file system housekeeping for you.
Uh, if you do a VG scan minus V.
It doesn't really tell you much more. Um… If you look in…
You'll see here…
This is a backup. What I… when I did that LV Extend.
The last time I did one. Um, this is creating, after an LV Extend of VG01 backup.
So, it was run on this host, the creation time was that.
Uh, what else? Um… VG01 had this.
This is everything it takes. To recover the logical volumes structure off the disk drive.
If I were to lose all the other information about it.
So, let's just say… the extend failed. I could recover the system by restoring this.
And I'd have the file system before it was extended. This can be a real lifesaver.
Lvm backup, LVM.com, the last thing I'll show people. Um… you have…
This thing, this filter. Um, the default one is STUPID!
It's this. This filter says when I do an…
A VG scan. What devices.
Should I be looking on. In order to find.
Volume group, to find physical volumes and volume groups. So what this says is, add any device that matches dev NVME blah blah blah.
Add any device that looks like dev. Md anything.
Add anything that looks like DevSD. You know, a partition.
Remove everything else. This saves me from searching…
All of the DVDs and CDs and tape drives and TTYs and.
Keyboards, everything that won't contain a physical volume. This says, pass it through there.
If you have a large system. Especially if it's got something like, uh.
A CD or a DVD rack. This can save you minutes in your reboot.
Um…
Uh, do you have an exclude line? Okay. Um, I've run…
That's this R. If you do an A, it will add it, and then this says, add this, add this, add this, remove everything else.
Yeah. The… tie… another time you want to remove things.
Is when you're dealing with sands where you have multiple paths.
So you might have SD A, B, C, and D. Separate paths.
Yeah. Well, or, um…
Nfs, um…
You shouldn't have multiple paths with NFS. But if it stands.
Yeah, well, but I'm saying, if you have NFS, you probably don't want to do a VG scan of something over the network. It's going to be presented to you as a volume anyway.
Um, you know, here, I can take my CD-ROM out of it, remove the CD-ROM.
Hang on, let me finish what I was saying, please. When you have fiber channel sands.
Oh, sorry, I thought you were up.
You have paths for each controller. On the host and sand storage unit.
But for redundancy, you want to use the multipath device. So you add in your filter Dev Empath.
Search those, and do not search… the specific dev SD devices.
Now I'm done.
Right. Oh, the… well, the nice thing is I can add the things I know have these, I can remove everything else.
And this is usually the safest way. If you start adding everything and removing what doesn't apply, you'll start searching DevTTY or something. It just…
This makes it much, much, much faster. And in fact, if you knew, for example, that all your MDs didn't have any logical volumes on them.
You can just add the NVMEs. If you only had MDs.
You put that, and don't search any NVMEs, because you're not going to find anything there.
If you did find a signature, it would be a duplicate of something that came from the MD, and you don't want that.
So this… this is… comes in a particularly…
Stupid form of… DevNBD star.
Um… And you'll… you'll almost always want to edit this and change that filter.
Uh, the other thing, I think that's… You know, there's all kinds of stuff.
You can configure in here. Caching metadata, all the rest of it.
It isn't a bad idea to go through once. And see how and why these are set.
You can find with these… with all, you know, CPU threads, hash zone rates, all the rest of this.
If you're dealing with performance. You know, a high availability cluster, a high throughput.
Reading that can be one time. Can be really useful. You tweak it once.
Make comments in here about why you did it, and then you'll forget.
You know, once you… once you tweak that. You don't really play with it again.
You might want to glance at it about every 5 years.
Yeah, well, yeah, it's… it's not a bad idea if you see… Uh, well, I mean, again, I use Gentu.
So, I'll… I'm gonna get a… if… If there's an update to this.
Um…
Gen 2 doesn't overwrite your config files. Um… Not at all.
But I might have something… I can do a diff minus U.
Of Etsy SU Doers. With Etsy…
There are other ZFDSU doers… Um, oh.
Um…
If I… if I neglected to update this, and there were multiple updates to it, I'd have CFG12345.
The assumption with 4 digits is you're not going to have 10,000 of these things lying around.
In the normal Y assignment system, which is… Pretty good estimate. So I can… I can look here, and I can say, okay.
By the last generation of change to my SU Doer's config file.
Updated the secure path. By adding this garbage onto it.
Um… it doesn't like my using the wheel all, because wheel is normally commented.
And I just did that. Um.
There's a change in some white space. But this is… this is what I would have gotten.
I would have taken this out and commented it. And I don't want to comment that.
So, on Gen 2, what I would do is run this diff minus U.
Grab that much of it. Make a patch out of it, do a patch minus P0 on it.
Um… But I can tell…
If… again, on Gen 2.
I can do a find slash Etsy minus name. Items for a seat.
These are all the things that I should go through and clean up. Postfix has had more than one update.
None of these is Etsy LVM, so I'm not gonna pay attention to it.
If this ever said Etsy LDMLDM.conf. I would do a diff minus U.
To see what they thought they would have changed. I can decide what I want, make a patch out of it, directly edit the file.
And then wipe out the config update. That's one reason I like Gen 2, is it doesn't step on my config files.
Anyway, um…
Did this make sense?
Anybody still awake?
Yeah, this is Randy. I, uh, I like what you covered.
And being a very visual person, where I miss out is being able to take what you did.
And you did a lot of good stuff that I really want to know more about.
Ah, then what you'll do…
If I had the visual to go along with what you were doing as you're doing it, it's gonna… And you're… you're talking basically about.
Your particular situation. So I don't have any knowledge of what that looks like. You do.
Well, what you do is you go back. Oh, God.
And so, consequently, you go through it, it makes sense. It doesn't make sense to me yet.
Well, what you do is this, um, hang on.
Because I don't have visual. But I liked the presentation because of the detail.
Um, if you go here, speakerdeck.com slash me. Um, you will find…
In the same place that Gary did, click! The whole… the whole talk is online.
I'm gonna have to clean a few pieces up and re-upload it.
If nothing else, this little white stripe across the top was a screw-up in the way it was laid out the first time.
Um, but… The… you can click your way through it.
The real thing to remember, and there are really good examples of this.
If you look up… oh, I shouldn't have killed that. If… and then I gotta go, because my wife is telling me I go ate dinner.
Um… If you look up.
Linux, LVM, FinPool.
You know, there are… mixconfig.org.
Red Hat… the Red Hat stuff usually tells you what to click in their menuing system to do it.
But at least it will describe what you're doing. Um…
They're very thorough, I know that.
Linux.org has got a good one. I would say the Linuxconfig.org.
Is a good place to look up. Just about… everything on LVM, because it will show you that.
The… If you go to… if you look up…
Linux, Unix… This is the book, I pasted the link to.
Um, you can… find it on Amazon for a variety of prices.
Boy…
Humble Bungle currently has a deal going where you can get it, plus a whole bunch of other things for $25.
Find a copy of this. And read it.
They… they stopped producing at Evie Nimeth. Was interesting, one of the… one of the relatively few women.
Who was heavily involved in computer science and Linux for many years, Unix for many years.
This was originally the Unix system administration handbook. Um… she vanished. He was doing a long sailing excursion.
And Bo vanished, so no one knows what happened to her.
But Nimeth, Sneebass, Snyder, Hein. Uh, the 5th edition included Linux.
This is an excellent book. Lvm hasn't changed all that much in a very long time, and because it looks like HP UX.
It's very well documented coming from HP. This is a great book.
Ibm also has some good documentation from AIX, which is LVM-based as well.
Still locked today in.
Yeah. Um… what every super user should know, don't use SU.
The other place you can look up a lot of this, uh, if you look up Gentu.
Ovm. Gen 2 is the… oops, not on Amazon.
On DuckDuckGo.
Um… Because you installed Gentu manually.
The… the manuals for a lot of this stuff. Are really good.
For the… for actually doing it. So, this is… these are the device drivers, this is what you need to have in your current.
This is… the eMERGE, okay. Uh, your mileage may vary, but you're gonna have to get LVM.
And then… it's unit.de Lvm start, you can rc update it.
System CT Eleate. Um… you know, my…
The knit script I've been using for… 20-plus years here.
Um…
So, what this does… is it wakes up in the morning.
You'd install BusyBox, does a sync. It mounts the…
The file system, the non-device file systems. It begins MD Adam does a VG scan.
Changes all the… logical volumes to active.
Mouse the root volume. Finds the init.
If there exists an init, it does a switch route. At which point, I'm running the init.
And I switched it. Otherwise, it gives me a nastygram and dumps me into a shell.
There's a copy in here.
Been busy box, and there's SBIN. Govm.static.
I compile that when LVM2 came out. It hasn't changed since then.
Um… matter of fact, that same in its script has gone through.
I don't know how many different servers. It just never changes.
Because I use a SIM link.
Okay, that's what I'm gonna boot next time. At that point.
Um… There we go. So…
I've got a menu entry. Normally, I boot NVMe 7. If… I want to boot in single user mode, I picked this one.
Notice how much text there is in each of these entries.
Does this look a little easier to read than what you get when you run grubconfig?
Uh, the trick here is there's an in-smot of LVM. The GFX payload avoids problems…
With the display not working. Uh, GZ GPT partitions, LVM, and XFS.
This is what it takes to read. File system. And that Insomata LVM is what allows me to boot off of LVM.
It finds a file system UID of this, which it generated for itself.
And then… it finds… Uh… slash VMWin is this this, because we've been using a boot volume.
And the root is dev, mapper, whatever. This could be dev VG00 slash root, same deal.
And then it has a slash and NRD. I could use my old init.
Um. By just having init equals… slash init. And it would switch to using that one.
The cute thing is, there is… A hard link to the last kernel that booted correctly.
So that I can't accidentally delete it. So, if I… if the last kernel I compiled fries me.
I can boot stable, and it will boot from VMlinus.stable. Which is whatever that last version was, I don't care. It's just a hard link.
If I boot the rescue. Um… there's an… that initrd.rescue, what it is is a copy of that old init script.
That doesn't do a switch route, it just leaves me in busy box.
So, if I toast the kernel. I can boot an old kernel.
That works, if I can't… if something's wrong with the mount points, or slash, you know.
Something else got screwed up, I can dump myself into a copy of BusyBox.
And fix it, and then reboot.
But I think these are a lot easier to read than what you get from GrubConfig.
They don't have all that extra stuff that you don't use.
All those minus-minus, you know, source equals, destination equals. Those are for graphical tools that are supposed to automate this stuff.
That nobody uses.
Um… But then again, if you look, um…
So this is inode 146.
So, 146… is stable. As a matter of fact, I did delete the old version of the kernel, which was not very smart, but I did it.
That's why this is a hard link, not a SIM link.
This always… I accidentally deleted an old version of the kernel, because I thought it was clean.
And then I realized it wasn't. And so, who cares? I can still boot this.
Um… And yeah, I should clean this up. The VML is 6.6s are…
Obviously ancient, um… This is up to 6.12.
But then… this works with LVM.
The root volume here. The VG scan just installs that.
And that's where that init comes from. If you use grub.
Where you're using LVM, that it knows how to read the LVM file systems, and it can find.
The kernel, and it can find the init, or initrd. Because it knows how to read the OVM systems.
Anyway, did any of this make sense? This is… this is paranoid, but this is something I've used.
Yes.
Especially when you get a new computer. And you're trying to figure out what works on it.
Uh, having a SIM link to the last kernel that worked properly.
Is a lifesaver. This .BD on the end. Um…
Is this, this local version. And what I do is, every time I go in.
And I say to myself. I want to add a new device driver, or I want to take one out, or I want to tweak something on here.
Because it seems like it's… it makes sense. I will up this. This is the first time I create this on any machine, it's AA.
And then A, B, C, D. So, obviously, I've made about 30 changes in the kernel since this thing was installed.
The point there is, I can have multiple versions of the same kernel number.
With these local versions. With changes in them. So if I'm trying to figure out why some device doesn't work.
I have BD, E, F, G, H, I. I can keep advancing it.
And one of them finally works, and I delete all the ones that didn't.
But it means if I create one that doesn't, I can always go back one version.
By just typing something. In the grub command line.
I don't know if that made sense to people, but… This is how I keep from shooting myself in foot when I'm playing with the kernel silver.
I do something similar.
Yeah, I think that the local versions, everyone's got their own, you know, 01234, I just use letters because it seemed reasonable.
Um, I stopped using single letters because I ran out. You know, I tweaked the kernel that often.
Um, but… Combining that local version.
With that hard link. To the last bootable kernel.
And having that in my grub config. Makes it really easy.
To go, oh, that didn't work, and I just boot the last bootable version. Done.
Anyway, um… But yeah, if you go to Linux config.
If you go to the Gen 2 LVM. It describes… Using LVM, it describes.
Configuration file. Uh, extending, reducing…
All the LV creates, displays. You can… there's a lot of working examples in here, how to create a thin pool.
So you can go through all of these and see. Lots of examples with lots of… text in them.
Anything… I would say anything you want to do, if you look up.
Gen 2 and that thing. It will probably be better documented than any other system.
Because it will show you everything from the kernel up, because that's the way you build Gentu.
You read it once, you'll… you'll forget it all. But it'll have made sense when you did it.
And you probably did the right thing. Um…
Booting Gen 2 from LVM… There's actually one really last thing, is there's a good.
Article in the Gen 2 wiki on how to compile the kernel.
As a generic user. Instead of being rude.
I'd recommend everyone read that, because it makes it so much easier. You can be yourself, you can build and install a kernel.
Booty. See if it doesn't work, and back it out without having to do SQ on everything.
And it makes you a lot less… vulnerable to RM.
Stars, you know, slash space star mistakes. If you're doing it as yourself.
You can't fry the existing kernels.
Um, and that's…
So, I'm gonna… I'm gonna clean up a few mistakes I made in here, and re-upload this.
Probably tomorrow, if anyone wants to look it up on SlideShare, it'll be there.
Don't all sound so happy at once.
Fantastic. Thank you, very, very… Thank you very, very, very much, Stephen. That, uh, really been a good tour de force. Yeah, a lot of fun.
Yeah.
Right. Did… Well, but… did this stuff make sense?
Very extensive.
Yeah. Mm-hmm. Mm-hmm.
Yes.
Yes, a lot of it makes sense, yes, certainly.
Okay. Nobody remembers the syntax. Look it up. But that Gen 2 thing on LVM shows you all the right syntax for thin pools and all the rest of the stuff.
Yeah, yeah.
Um… So yeah, I've got to go through and… find some errors in this thing that I made.
Quick hacking it to add some things, so… hopefully tomorrow we'll have time to make these changes, and then you'll see a new version of it on speaker decks slash Lembark.
Hmm.
Hey, Steven. This is Randy. I'm wondering, um. Uh, how good is the man pages for LVM?
Mm-hmm. The man pages are good. Uh, the thing about them is they're man pages. They give you the syntax.
They're wonderful reference, but not great for learning.
Yeah, so you can… you can find, you know. Sure, you'll know what all these… you'll know that these are there.
But… it's not gonna tell you how to do it right.
It's going to give you a few examples and go away.
Okay. Okay.
If you go to the Gen 2 LVM wiki page. It shows you how to do it right.
If you go to the Unix and Linux System Admin book.
Okay.
Or you go to… that, you know, Linuxconfig.org.
They're going to show you the… a lot more detail about how to make it work properly.
The thing… the thing I would… I would do for most people.
Okay.
Properly…
If you understand the basics of LVM PVCreate, volume groups, logical volumes.
On a notebook or something, you create one physical volume for the one disc, you're gonna create one logical.
One volume group for the one disk that's got root in it. You create logical volumes for things.
At that point, the real utility of it. On a notebook or something for your day-to-day use as snapshots, so you can back things up properly.
Or, as logic volumes. Backing up, uh, or backing virtual machines that you run on it.
Yeah. Uh, the other thing you'll use it for is…
Well, Steven, I…
Again, you can… you can do things like have. File systems that run away.
Uh, Vartemp. Var log, home.
Where you can get things where the file from hell accidentally fills the file system and locks everything up.
You can make it so that those file systems are separated.
And you might lock up the logging system if our log fills up.
But you can still get into Super User and, you know, nuke the logger.
I actually have a small nitpick with that. I would say you can easily.
Do what you just said.
Yeah. The other thing is, thin pools are cute, because if you have things that do a lot of writing.
All at once. You can have something that is still killable, even if it fills the file system.
Which has saved my neck more than once.
But Steven, again, thank you for, uh, for the presentation, and I'll be looking for your updates.
And, uh, I've got to go for another meeting.
Yeah, everyone. If anyone. If you look on speaker deck, if there's any of the stuff I've got there.
That looks interesting, uh, warrant Gary, I can give them any time.
Have to write them over there.
Yeah, I didn't either. Okay, uh, also, I'll mention to you, Steven, uh, uh, Dr. Timothy.
Uh… uh… bylash, uh… hope I got the name right, uh…
Anyway, yay, alright.
Really help…
Yeah, that's pretty good. I'm sorry about that, I just… I'm a physician, so that's my usual Zooms, so, uh…
And I'm in the St. Louis, not San Diego, but I am so swamped.
That's fine.
I understand. Anyway…
Uh, doing… I'm kind of an eclectic, and I do… do we have time for any comments or questions?
I think so. I'm just gonna point out to Steven real quick, give him a big thumbs up.
I got… I got a couple of minutes when the wife is gonna strangle me if I don't make dinner soon.
Well, if you got a question, go ahead.
Well, I, you know, there's a… I think they're really good. I'm trying to get into Linux, because I'm a Mac user.
Now, if you're a Mac user, you're using… you're using BSD every day.
And it's many, many decades since Unix for me. So, and it says…
Well, okay, and… but more and more, we're going to the… Well, yeah, well, but it's…
Get OpenBSD and be done with it.
The utilities program is pretty nice if you're formatting a, you know, a physical device.
Right.
It has dynamic reassessing and things as I use it, and then… So I've used it mostly for backups and things over the years.
Um, I'm allergic to Windows. And, uh, the MAC system is introducing so many bugs.
Well, we have two of us.
I've gotten sick of macOS for a lot of reasons. They've also locked down a lot of them. You can't do anything with the systems.
That I spent… Right, but…
Well, this is the issue. They… they… it's a little authoritarian.
Um, but I am not really good at a command line interface, and so… I like graphical interfaces, and…
Well, then if… use OpenSUSE. Um, the…
The graphical interfaces to play with the system. Are… are good.
Uh, YAST is going away after many, many years, but it… it works.
The things that replace it work. And… I'm done. My wife has just informed me that I'm finished.
I would… I would do this. If everything that I did tonight.
Okay, well, thanks… This is a very… it's…
If you pick whatever distribution you like. There will be LVM management tools that are graphical for it.
And you can click the buttons. The difference will be you're going to get things like a partition that pre-allocates the entire flipping system.
When the smart thing to do is allocate the least amount that you can to everything and grow into it.
That's a good suggestion. I really… this was very interesting, so thank you.
Because it's so easy to do.
Um, and again, if you care about getting into Linux, read that Linux and Unix System Administration Handbook.
It is flipping thick. It's many six-packs to get through it.
Go through and read it once. You'll forget 90% of it.
But something Freudian will kick you and say. This is dumb. Why am I doing this?
That's one of the… that is still the best place to start.
Okay.
Hey, uh, I know Steven is on the East Coast, uh, so he's an hour ahead of St. Louis.
I'm out of here. I've got to leave now, or my wife will kill me.
Thank you, Steve.
Yeah, that's what I was just gonna say. I understand you gotta get going. So, uh… even go with a big thank you, and I'm just gonna make a couple comments to those who are still with us after Steven gets off.
Uh, uh, first off, the questions and the. New things you were just presenting to Emily, uh…
Very good, we love it, and by all means, I'm once again going to encourage you and anybody else.
If you're not on the discussed mailing list, please sign up for it, and post some of these things there, start a little conversation there, and maybe we can find a speaker to follow up on.
On various points along these lines, so, uh… Everybody stay in touch.
Sure, sure. I do, and if we have time at the end that's not related.
It's a… just a direction to go in, so I don't want to interrupt what we're doing.
Oh, but yeah, like I say. Even in discuss, uh, that flips a clue as to what directions to go down.
So, I… I have a problem because I, um…
I use a lot of, uh… I use a, right now, Thunderbird as an email, uh, program on a desktop, and everybody's going to apps.
So, my… for a particular website I have, the…
For various reasons. The emails is unrelated, but the emails are on one server system host.
And I want to nap… I want to move them to the other one where I have the websites.
And so, I backed up on Thunderbird. This is my current urgent issue.
And I have all the emails. If I go to the archive folder that I archived.
But I can't see them in the individual folders. And I'm afraid to move everything.
Until I'm sure I have, though. And I don't know where to go, who to talk to, it's, uh, you know, it's…
Well, don't forget that 5 days from now, the new Linux group will meet, and you'll get some help there.
That's what I was thinking. I wanted to ask that, if that makes some sense.
Uh, because I think…
Yeah, I mean, that should, you know, that's the kind of stuff we can deal with, and I can often… sometimes provide individual help, if you can trust me, but…
I appreciate that. I'll see if I can clear that schedule on Tuesday. I will go if it's free.
And I missed the last one. Oh, no, if I'm free.
It's free. All our meetings are free. Oh, hey.
Yeah.
I have some conflicts sometimes, but this is great, and I appreciate the… I am, uh… very critical of the operating systems, period.
And, uh, I was at Bell Labs for a while, so, um, I knew some of these guys. I wanted to thank you for the Kernighan video.
That I looked at, I think, was sent out on the emails.
Um, because, uh… I think we… all the operating systems, I think, are the problem, and I think we're dancing around the issues sometimes, but that's.
Probably the problem is… Actually…
That's just based on programming eclectically forever. And I… I'm using… I have to… I have to use text files more and more for compatibility, because I can't open files even. They think I have malware on my own files.
I don't know what to do.
Well, what things… what thinks you have malware on your own files?
Now, the Mac operating system? It gave me the… I brought a file on a USB from my 10.8.
Mac system on a laptop. And it said, this file may contain malware, we suggest you delete it.
And one of the… when they upgraded to 15.1, I couldn't sign in.
Wow. Uh…
I had to go around and kind of, uh… actually, it was a Best Buy here in St. Louis, help me. Uh, you know, you run the utilities in a safe mode, and then you go back and forth, and eventually you sign in.
And if you didn't shut off the computer, you were fine. So for a week.
I couldn't shut the computer, and then the 15.2 update, I could sign in again. Some… there's… there's… something that happened, I don't know what it was, and it's gone away, but they do that.
And because I have a lot of older programs and ways to do things, and files from, you know, 30 years ago.
Uh, it doesn't like that very much, especially in PDF formats or things.
I just mentioned that, we'll look forward to meeting that group.
Uh, as a… as a… as a… so I think I need to go for the internet anyway.
To a Linux system.
If you can't… if you cannot make it Tuesday, uh. Join the Discussed mailing list, and.
She kelp there, and… maybe I can help, or somebody else with your problems.
Okay. Alright, I'll try to post it, but you know, I think my issue, again, the command line.
Well, uh…
Is, uh, difficult for a dyslexic. And I really like the Mac, uh, graphical, because you have an icon.
Represents anything, and so you're not…
Well. There are… most… most all the Linux distributions now have some form of graphical, uh.
Really? Really? Okay. Okay.
Thanks. Yeah.
My mother-in-law just switched from Windows 10 to Linux Mint Distribution.
Loves it, and she doesn't do a single thing on the command line. Everything is graphical.
Amen. That's a…
Yeah, I… that's the one I was thinking of going to, but again, you know, even just choosing the distro is difficult, so I appreciate… Yes, sir?
Yeah, Timothy? Steve Stegman here. I'm… getting ready to install Linux Mint on a…
Computer for a friend of mine, and… She's been using Windows in her phone, and God knows what else.
And I'm gonna be in the process of teaching her to use Linux Mint.
I've got an old book that I bought… oh, God. Almost 50 years ago.
And, geez, these numbers are getting big. Um.
Double-digit years.
And the first half of the book. The book… Pardon?
Well, you might as well say 60, Steve. You might as well say 60.
Well, you know, it's just… it's just like… We just invent new numbers for the bigger numbers and make them smaller.
You know… But, uh, at any rate.
So, you're…
I got a book… oh, God, I… it's in my library, I'd have to go get it.
Okay.
But, uh, I can… I can put up, uh, on, uh, discuss…
I think it's still in print. Strangely enough, but it's…
Yeah. So… So, I get… I get emails from that, I believe. Now, are you talking about going to a particular place that posts stuff?
Uh, no, um… That's our mailing list.
No, it discusses our mail list… email list, Timothy.
Right. Right. So I get that. So… then, uh, uh…
Okay, well, I'll put it in there, and the first half of the book.
Okay.
Is how to… it says how to run Unix, okay? Not Linux.
Because the book was written before Linux. And… the thing is, the second half of the book is like a, um…
Uh, dictionary of… Unix commands. Now, the funny thing is.
Or maybe it's not so funny, um… all of the commands… It's all based… that's all based on command line, but…
Right.
The… the thing is, those commands, I've got a Linux… one Linux system that I just upgraded, and it's on the latest.
Latest and greatest Ubuntu version of Linux. They all use the same kernel.
And the commands that you typically would use, you know, to play with your file system, look for files.
Edit files. Those never change. They're exactly the same as they were back in the early 90s when I used to… learned this.
They haven't changed. They don't change. I've got… I've got one computer sitting here with.
I'd say the operating system, it's from 2000… 18?
And I just upgraded to 2024 on my second… I've got two on my laptop, or my desktop.
And… I'm sure the new one is more secure.
But as far as using it goes. Somebody redrew the icons, I may never forgive them for that.
But, you know, all the stuff is there. It all works the same.
That's the beauty of Linux, or, well, Unix, Linux, whatever. Um, is it doesn't change. Once you learn something.
I've been using the same commands. Since the early 90s.
Is that early 90s or late 80s when I first learned it? I don't remember.
It's too long ago. But the stuff doesn't change, and… I highly recommend working on the… well, working toward the command line.
Because you're not gonna find a graphical interface for everything that you want to do.
Right, right.
And the command line can do anything.
Well, to be frank, the reason I use it is because I like the desktop, and I use it as a database.
Yeah, okay.
And I do, because there's a conflict. Between the different formats for letters and numbers, the Mac.
Treats numbers as numbers. So, the Unix system, I believe, is the same as Windows.
201 comes between 1 and 10. Because it treats numbers as, um, as, uh, letters.
Are you… are you talking about sorting a file based on…
So… Yes. So, I… my filing system for the desktop is…
Two-digit year dot, two-digit month, dot two-digit… day. And that allows me to list by title, have a title, plus when I.
Yeah.
You know, labeled the file, puts it in chronological order for me if I want. And so… It's… I can't do that. I haven't figured out how to do that. With the other ones, when I send to Windows people.
The file order I sent it in very nicely gets jumbled, and they don't understand why I organize it that way, so… I… I… I… it's actually an integral part of how I do, and now.
Because of the, I can't even open, you know, Appleworks files.
Um, I use text files exclusively for stuff, and I don't even have a contact manager.
So, uh, you know, I just find that, uh. Even my calendar program is cross-platform. It's one called BusyCal, I think.
But all those little details, I'm sitting here running a computer system instead of actually doing what I want to do, so I'm really afraid to leave the Mac, except.
It doesn't work sometimes.
Well, I… I don't run Mac myself, so… you know, I didn't know they were doing all this.
Um, it sounds like they're getting too big for their britches, sort of like Microsoft.
We know everything, and we will tell you what to do.
Yeah.
We will charge you for it.
Great. Great. You know, they… when they have a bug, they refuse to tell you.
God, I hate that attitude.
They… you don't understand what I'm saying? As you said, you're reliant on the community.
And you're always playing… you're really paying… and they're upgrading the systems, uh, you know, like, bi-weekly sometimes, and all these programs.
And they're… they're really throwing… This is apparently a big security threat.
Yeah…
So, for a while.
I've got a Samsung watch and a Samsung phone, and… They used to work together really well, and the last major update just totally.
Just totally. Mmm. Messed it up. And… I'm not gonna go out and buy a new watch, I'll just live with it, because I'm too cheap.
But… That's one thing I really appreciate about Linux.
Any Linux system that you got. We're Unix, that works.
So, yeah, and to, you know, to… to do things, like, for… I'm putting up a new, uh, small website.
And, uh, I'm using a theme in WordPress, but I write my… my web pages really in HTML.
Because WordPress is a kludge, even they're just their interface. And I'm looking at this, I cannot… I cannot figure out how to substitute my… photo in the header, you know, they have a logo.
Okay.
I made a logo. I can't… I'm, you know, it's two days, and then it turned out I didn't have enough space, and I weren't looking with tech support, and it's… where it was temp files all over the place, and I'm sitting here 2 or 3 days of just doing computer work for something really simple.
So, all these other technologies are beyond even just the operating system.
How to integrate. So, anyway, I'm really… thanks for the time. I will… I will try to get to that Tuesday, uh, New Year Linux.
I think… I think that's where we have to go, and Apple is too big for their wishes, and Windows is like a sieve for me, so…
Yes, I, I will, I will take the time to do that. Great suggestion, thank you.
Yeah, Tim, if nothing else, post a question to discuss every once in a while, and…
Yeah, and uh…
Hello? I wanted to, you know, the idea of virtual. Uh, I think is what killed the Mac operating system, because they couldn't figure out a way with these icons.
Moving around from a system that's across the telephone line. The delays are just… killed it, so they went to Unix.
And it's, you know, underneath Unix, but these systems are not good enough, and just to do a plug.
There is a… I use a database on the Mac that doesn't run anymore called, uh… it was Helix.
And then QSA… Uh, on it. Now, they're revamping it. They have a server that runs.
Cross-platform, but the actual database only runs on an older Mac system, the 1012 is the last one, and it was a graphical interface, had a metadata structure.
And so I wrote all these, you know, my bookkeeping program I use every year, go back to the old machine.
Um, and so it was interesting, a different paradigm. But it… they… they stripped everything from the macOS. It does… never ran on Windows, and… and so…
That's my… I come at this as a kludge, as a kind of eclectic, and I thank you for the time.
Oh, thank you for all the good comments, and uh…
Yeah, we'll, uh… I'll post that book on, uh… Uh, discuss, and…
Great.
I think it's a… I'm pretty sure it's available on Amazon. It's an old book.
They probably updated it.
Yeah, and I'll get that… I'm gonna get that bundle, too, and I think… but, you know, I… again, time is an issue.
And, uh, there's lots of other things going on, too, so eventually that has to happen, but… Um, yeah, so that's why I'm here to learn.
Well, you know. It's going to be painful to switch off a Mac and onto Linux.
But the thing is, the Linux isn't… they're not gonna… pull those kinds of stunts on you.
Okay, I'm here, that's recording. Keep recording.
The community just wouldn't tolerate it.
Right. Right, it's much more stable at this point. Um, and…
Well, it is.
And the thing, I think, is that everybody's forming their own, uh… hard drive in the sky, and then putting a fence around it.
And I'll tell you what I think the real issue is. Putting cell phones in requires the use of HTML.
And so, Apple used to use TIFF. Windows, I think, used PNG. I'm not sure.
Um, but TIF was great because the operating system did all the work, you didn't have to program it.
Now they're all using JPEG. And, uh, but the HTML is not… is not standardized.
And so you spend your time, all these systems are divided, and security becomes a big issue, I think, with the cell phones, so… I'm trying to look at it from the holistic point of view as well. As a database users that are crashing… that are crashing the hospitals open.
Sorry, I could talk about a lot. You stimulated a lot of thoughts in my mind, so I'll shut up now.
Well, we'll start posting on Discuss, and I'll put your name in there when I post that book, which.
Yes.
If I can find it tonight, I'll post it tonight, but… It may or may not be able to find it.
Okay, this week will be good. Yeah, just don't… don't rush for it. I appreciate the input.
I'll make the point that, uh, one of the potential speakers I was trying to get, but I haven't got him locked down yet.
Uh, but hopefully maybe one of these future months. He's a guy who's really a Mac heavy. He's really an expert in that.
Uh, but I was trying to get him to kind of come and… We had them a few years ago, basically do a presentation on.
Yeah, here's the underlying… Unix aspects of macOS, and.
Okay, it's really… and I… I called a tech person today about this Thunderbird problem.
What works, what may give problems, and uh… I was hoping to get them to do a follow-up on that after a few years here, but uh… I haven't gotten to commit yet, so…
I went to Best Buy, I went over to micro… Computer Micro, whatever that is, on Big Bend.
Um, nobody has the cross-platform experience that I've access to, and.
This is Stan. I have, uh, duplicated my email over on another.
So…
Position that I've got, and it's really not much to it. It's a matter of permissions under your Unix or Linux, or whatever you're running.
Half those guys at Best Buy, they're just trying to sell you something.
Yes, yes, but I can't get it… yeah, yeah, yeah, yeah, yeah, I hear you. It's… it's thing… it's… you know, one of the…
That's true. No, I understand, but I… Yeah, yeah.
All right, and even Micro Center, you know. There's only one guy at Micro Center that knows anything about Linux.
And he wasn't there.
Yeah, no, he wasn't there. So… Yeah, yeah. He's there, like, 2 hours or something. But this was, uh, um, Computer Solutions I called today about trying to go see them.
And, uh, you, you know, you freak people out because, uh, we live in our bubbles, and I am… I've been… because I just programmed so long, since 1970.
And so, there's just too much stuff rocking around my head, and too much I've forgotten, so… and I'm… I need to make it work, so I'm trying.
Well… Yeah. Well, there's… there's, uh… you gotta remember that a lot of times you'll go someplace on a… for a Linux users group, you'll go and ask a question.
Yeah. Yes! Yes!
They'll go to the command line, because all the different distros, their graphics are, you know, are different, so that… So, but all the graphical systems are underneath, they're using command line stuff, so the easiest thing to do is solve the problem.
I know. I… I hear you. But then… but then I go on… it's… you go online.
From the command line. However. I… I… I… I… No, I… I try to… I need to remind people that when they're coming to Linux.
That the graphical systems. 90% of people, they don't, you know, just a normal user.
Does it need to use command line. It's only for the very specific problems.
Kind of thing. And there…
Well, this… yeah, this one, you're going into the, uh, you know, where the emails are stored. It's changing… my system is changing permission… Right? It's… it's putting no access for general users, right? I don't know what it's doing or why, but I can't, like, download it and reload it, or get a new computer, or… I just… it's kind of like I'm frozen, I'm afraid to do anything, because I don't want to lose what I have.
So, I'll look forward to the discussions. I'll try to…
Yeah, um… the good news is. If you… if you got an old computer, you can probably run Linux on it. It doesn't have…
Yes. Yes.
It doesn't need the resources that Windows does. I don't know how much resource MAC needs, but I suspect it's a lot with older graphics.
I… I…
Oh, it's… it's huge. Yeah, I bought… I did buy… I have a laptop that's sitting and waiting.
I… Okay, I highly recommend that you get this PDF.
Oh, okay.
Oh, sometimes the buy-buy Windows 10 PDF. The first half of the book talks about.
Yes.
Loading Linux Mint and using Linux Mint. The second half talks about Windows stuff. Now, you're using Mac.
So that's based on BSD. So if you read the first part about Linux, you're going to understand 90% of what this guy is telling you.
That's right. But, you know, just as an aside, to change the owners.
Yes, you can.
Yeah, that's… that's actually, uh… Tim, that's actually very easy.
Of files, you can't really batch that. You have to do a file… Well, if you like…
All right.
Um, I've done some support work for customers. Um, and… what I found was they were using Macs and things, and they're fine.
And I called up the command line. You can call up a command line on a Mac computer.
And what you find out is you… you're staring at a Unix command line.
Yes. Yes. Yes?
And that's the same as a Linux command line, and nothing… you know, the commands that you normally would ever use are all the same as they were in 1990. They don't change that stuff.
No, but I can't… they don't… they don't let you get into the folders, even, much less find the files.
They just hide everything, they move it around on the next distribution.
Oh, God.
And you're… everything online is out of date, or, you know, and now I… it's an M1 Mac, it's not the M2, I should have gotten an Intel processor, I would have been happier, but… you know, that's the way it goes right now.
Maybe one of these days.
Well… If you got… if you got an old computer, in fact, I probably got one around here, I could give you one.
You know, that I'm not using.
As I said, I just… I hightailed it out of California. I'm trying to work as a doctor, actually.
That's a whole other story. The databases have crashed the hospitals, you know, the epic computer.
It is $100,000 for a installation in an office. And… and… That's the small one.
Oh, is that what Epic charges? Jesus, God.
That's the small one. It's up to… it depends, right? That's the one that's virtual space.
Kind of just looking in. Yeah, it's…
Oh, that's the one that's running on their little cloud.
Lee, we're in the wrong business.
I've been doing this for, you know, collusionally for a long time. I write databases. It was fine, and then right now… anyway.
My priority is putting up a billings module and a schedule module on a website.
So, uh, trying to go online for that. And it's… it's been, uh, slogging. So anyway, that's off-subject for this, which is virtual.
But, uh, I will look forward to the New Year Linux group, and I'll get there. And I will post. Appreciate the comments.
I just, you know, I worked with these guys at Bell when they… this… I left there in 90… in 81.
So I was there 77 to 81. When they were breaking up the lab.
And, uh, I actually worked on PDP computers before that. Uh, got involved when we went to a VACS system with those people, putting that up, so…
You know, and they're… the shell is so powerful, but, you know, I knew a few guys that went loony doing shell programming.
So, you gotta be a little careful about the whole thing, right? It's really, in a virtual world sometimes, even though it connects to real things. So I… that's… I just have too many different perspectives, being an older person.
And, um, still trying to make it work, and yet, uh, Apple won't talk to me, and my computer didn't work, so I'm looking for help. So I'll come Tuesday, I promise.
Well, eventually, what you probably have to do is export all your data from that op… database, geez, that's scary.
And you export it to… some number of files, and then you try to import it into something else.
We've got people here, not me, but we've got people with a lot of database experience.
It's gonna be… You're gonna have to dump the IMAP data, and you're gonna have to.
And…
What are you trying to go to?
Oh, I'm not trying to… I'm, uh, paper right now. I, you know, there's databases, high data. This is… so I'm interested in the… in the theory a little bit, and where we're gonna go, and now with AI.
We got real problems, because I do data modeling, and it's so… I'm really an eclectic. This is about programming.
And about using computers, and all I know is I have to replace the battery when it comes in.
Because it's out on my laptop, and after I do that, I really would like to back it up beforehand, but it's like… Well, I used to… I can't find the drive that I had the, um…
The Mac version, and I also use Carbon Copy Cloner, and it's like, well, then you have to reinstall each of the programs and put the passwords in.
And, you know, it's all… make work. So, I just try to keep things going as long as possible, and.
Yeah…
The internet, you gotta go to Linux. I just don't see… I just don't see Windows as surviving, and uh…
No.
Everything is using that, and of course, there's just an aside, um… what is that, uh, database company… oh, I just get the… and next time.
For next time. We'll talk. Thank you.
Postgres, if you're looking at a database, I would say Postgres. If you're needing something that's multi-user.
If you're just tinkering. So, if it's single user, then you should be able to sh…
No, I use single user, right? So… But it doesn't…
Use a SQLite file.
Sql… Light is a format.
No.
Yeah, but SQL is obsolete, in my view, as a database, right? So… This is where you… Unix is 75 years old. I mean, this is…
We really… what's that name of that company that does databases? Oracle?
You think at Oracle.
So they're doing well, because… Oh, no! You gotta tell me! But…
Puke!
Okay, so I don't know the details, I just know from the outside at this point, and we… but… It's a donut. I'll get the name next time. They're rewriting…
It's really interesting. You know, the word order and the bit order for Mac and Windows is reversed.
So you essentially… the problem is the communications, you have to do a sort.
A resort on it, which is… which is, you know, wasn't good for a while. There were some programs that used to do it.
And it's 64 bits, it's really hard. So, you know, there's all these problems about it that came across.
I'm literally… I use a text file when I talk to people on the telephone.
And I keep it in one file, I have it in one place, I used to put patients' number.
In each file, and then do a file find. On the Mac system, pull everything up on the patient.
And it's usually in one folder, and every… it's right there, and it's, like, instant, and all these other systems, you're like, oh, waiting, and, you know… So, data integrity is the other thing we didn't really talk about with these, um…
Well, you're elephant in the room there is HIPAA. You can text files, it's federally, uh, not permitted.
Virtual systems. Well, yeah. Well, which is designed… that was designed…
Yeah, well, well, everybody has the information, but we pretend, and that was designed to get rid of the doctors. We're not seeing doctors now, so there's a whole… Yeah. So, I'm trying to run a small, uh, consulting practice for menopause and for hormones.
And, uh, I don't need any of that. So, it's all overhead, and I don't… you can't charge… the insurance costs more than you pay.
Oh, well…
So, it's a whole other use of databases that's a problem, so… It's really fun, I like this. What's your next… what's next on the agenda here? So we don't…
Okay. Alright, I'll, uh, I feel like I monopolized too much of this.
Have to look at the calendar. Jerry hasn't told us yet. Uh, let's see…
No, no, no, we're enjoying it. A lot of us are retired. We do have more time than people who are.
Yeah.
Busy working.
Oh, good, good, good, good. Well, I, you know, this… I'd like to plug this, uh, uh, donut. I don't remember the exact name. It's the new implementation of Helix.
Um, and they just, uh, last year, they marketed a server.
But you have to run the database on a Macintosh before 10.13.
Uh, because they stripped everything out that allows it to run. And it was made by a brilliant programmer, originally called Helix Express.
And he died shortly after they marketed it. He was brilliant.
And it has a metadata structure, and it's icon-based programming. And they have things called abacuses. It's really interesting as a database.
But what it allowed to use to use is not just joins between.
Folders or files. But, um… every… every, um… Every…
Field, or calculation on a field in a table. Could be a key.
So, you know, the data integrity was taken care of by… they re… rewrote the graphical interface.
They dumped the Mac one, and they wrote their own graphical interface to do it. I mean, this… but then he died.
So, it's been limping along. Jobs didn't like it, so it's a… that's a historically interesting thing. I can't use my databases I wrote for the office, so I… I'm scrambling as an older person, still practicing, still trying to…
Yet the, uh… the world back to doctors instead of PhD computer scientists.
If that makes any sense at all, right? So…
Well, your problem is not PhD computer scientists, your problem is children writing software.
Well…
And bureaucrats thinking they know better.
They don't… they don't properly test anything.
Oh! Yeah! But that's the corporations, that's not the programmer. They don't have time.
It's not nose developers.
No, the programmers know what needs to be done, and they won't… they won't… if they want to keep their job, they can't do it.
Right. Right. And… and so, I mean, seriously, I was on with, uh… So, LabCorp, you know them, right?
So, I tried to get, uh, some, uh, labs for patients, two patients.
One of them I ordered back beginning of June. So I pull up the calendar… I look.
They haven't had it drawn yet, it dropped off. So, I went on their calendar.
I couldn't search beyond June 16th. There's a bug in their new, you know, they keep changing stuff.
Their calendar doesn't go back more than 2 months. You have to search themselves, and, you know, so…
Oh, God.
Yeah, and um…
These are bizarre, stupid… you know, they're just not… nobody's writing good code, and…
No. And everybody thinks AI is going to be the panacea, and it's not.
Particularly for the interfaces, and so… I, you know, I try to…
It gives the wrong answers, is the problem. When you do modeling.
Well… oh, I had an interesting statement from that one, from Amazon on that one.
It says…
The… one of the executives said, replacing junior devs with. With AI was the worst decision ever.
Right. The middle managers are gone. The people that actually made the interface between the ignorance.
That are rich, and the slugs that do the work, like me.
You know, and now there's no connection, so… and you know what I say? I mean, you… It looks like we have one big database in the sky.
And everybody's got a search program that's either proprietary or buggy.
Or, uh, you know, is blocking things. And so, I googled a chat GPT a couple months ago, it said I don't exist.
Literally. Uh, it's… it's better now because I bugged it a lot, but… Um, what… what are they searching? Where are they going? What are they excluding? Who is it distorting it?
You… I mean, I'm preaching to the choir here, you guys know.
But I'm… I'm still slugging that away, so… I… in California.
They were going to pass a law for state-level insurance, a state-level single-payer health insurance.
And they were going to allow me. Not to have to use computers in my office.
Because it's really just for Bill. And it didn't pass, of course, but that's the issue. Everything really is just commercial clerks and, uh…
Uh, and so you guys… You're really in the forefront to some degree, I think, in the battle to try to win this over again.
Because our databases are not working. Not working. For me.
We're gonna look forward to a lot more discussion on this, I think.
Well.
I hope so. I am starving, and so I… They… it's really great to have you around, so I'm really… I just found you, so…
You had asked… you had asked earlier when the next, uh, gathering of this group, and the next one is the.
New users, new user Linux group. Uh, it is next Tuesday, uh, evening.
Yeah. Yep.
Uh, I believe I have that correct, Stan, correct me if… Good.
Yes, it's Tuesday.
It's on my calendar, I get the link sent, I believe. I'll look for it, and uh… I will, uh… and it will be recorded.
Yeah. And if you can't… if you can't make it, you know, we can make other arrangements.
A question that… a question that I frequently ask at this point in the evening.
Okay?
I understand… I understand. I'm just…
Uh, for those of you who either have not been at one of our meetings in a long time, or this is the first time you came, either way.
Uh, what led you to tonight's meeting? How did you find out about it? What enticed you to come?
Timothy, I think, uh, we know that you're relatively connected.
You… I think I answered that. I was looking… I was looking for, uh, you know, groups, and I came across all your.
Club meetings, and… but this is the first one I've actually been able to get to, so… I searched for it.
Okay, where'd you come across it? So just a Google search, and it came up with us.
Yeah, basically, and look through. Um, I, I, uh, you know, I'm about transparency and making it easy for uninitiated to find stuff.
D.
And I find the older people can't actually search on the internet.
Yeah, okay.
Uh-huh.
And the young kids don't even know what it is. So, uh, we have an information dearth, and it's… it even… because I do a lot of scientific programming, I do a lot of medicine, newer endocrinology and stuff, I publish.
And what I… looking for information is getting so hard. Even if you know what you're looking for.
And, uh, it's really… there's a whole, as you say, there's a lot of stuff that could be discussed, so I really appreciate this.
And, uh, I do think that at least because it's open source, look at the open source Chinese.
Oh, yeah, yeah.
Gpt program. Right? There's… they have been gouging us for a long time by having us sip information through a straw.
And now, everybody else can do something outside of them, and I think they're really in trouble, so… Um.
I'm a fan of open source. I think in medicine. People don't think about it, but I do believe HIPAA was put in place to get rid of the doctors.
Because… because we need information. And if you restrict my information, then you don't need doctors, which is what happened.
Uh-huh.
Yes. This is Stan. Steve, I want to ask you before you disappear.
So… go ahead.
Have you… are you planning on using that VIN toy thumb drive I sent you?
Yeah, I'm gonna try it, I just… you know, yesterday morning, my wife fell, we had to call the ambulance, you know, she wasn't hurt, but, you know.
Okay. Oh, jeez.
It was just… it's just blah blah blah blah like that, and I've been… you know, we've been running back and forth to the doctor. I just haven't had time.
Okay. Yeah.
No, I'm sorry, Sam. Well, I live out in Wright City, so traffic.
Yeah, just the traffic alone.
Well, up until a couple of months ago, now that they start working on I-70.
Can we do a bookend, Gary?
First, I want to see if anyone else would be willing to answer that question that I just had Tim, uh, answer.
I mean…
Where did you find out? How did you find out about tonight's meeting? Was it the topic, the speaker, or just that it was one of our meetings that brought you here tonight?
And we can keep going after the bookend, but we need to do a bookend.
Yeah, well, we need to get this answer recorded. Anybody else willing to answer how they found it, why they came tonight?
You made me. Oh, okay, yeah.
Okay, Dolly, I think you mentioned when you first… when we were on earlier that you've been here before, but uh… For some reason, decided to come back again, and we're happy to have you.
Anybody else who hasn't been around in a while? Robert, you've been very quiet tonight.
Vincent has been here 2 or 3 times before, but hasn't said much.
Okay. Alrighty, I guess no one's going to answer that. Okay, uh, I guess we'll put a bookend on this. This has been the, uh, monthly St. Louis Linux user group, the St. Louis log.
Uh, today's date was Thursday, August the 20th… I'm sorry, 21st.
Uh, 2025? And, uh, our speaker tonight was Steven Lemberg.
And he was, uh, talking to us about, uh, logical volume managers, LVM.
Analytics. So, thank you all for coming tonight, and uh… Uh, see you maybe next Tuesday. Bye.