All right. Hang on, hang on. We're ready to record.
And I go, okay, this is the bookend. This is the monthly St. Louis Linux
Unix group, also known as the st louis lug
This is Thursday, the 21st of November, 2024.
Welcome. Our speaker tonight will be Stephen Lembark from Workhorse Computing.
Steven did a wonderful presentation last month at our Wednesday night general meeting
on a bunch of command line stuff in bash
And so he's kind of following up on that in a way.
It's not really a part two, but more just a similar talk
And he's going to talk about obeying the command line.
and uh but get
GitOps and TypeScript tonight. So we're looking forward to that.
But before we get to that, we have a few announcements and
also an opportunity for some open Q&A.
So let's see, as far as announcements go.
Just got to pick some stuff off the calendar.
And let's see, it is
And this past week here, I think yesterday it was, 55th anniversary of the first
permanent connection of the ARPANET, which is what became the internet.
And anybody know what that first connection where the two endpoints of that was?
Go ahead, push your microphone button.
Okay, which between UCLA and Stanford Research Institute, SRI, and Menlo Park.
So 55 years ago, hard to believe it's been that long.
something a little more recent, 10th anniversary of SUSE.
being merged into micro focus
Yeah, that was a big thing back then.
50 years ago was the beginning of the
DOJ, Justice Department's antitrust suit against att
that resulted in the breakup of AT&T into separate companies and
Spun off Bell Labs and of course spun off Bell Labs and ownership of Unix.
Sure.
see on uh
slug things coming up on the calendar.
Next week, specifically next tuesday
is the newcomers meeting.
For those of you who may not be too familiar with it, I'll give a very brief description. The newcomers group basically
does not have a fixed presentation.
It's basically just whatever people want to talk about.
And there's a little informal poll. They decide who the
newest user of Linux is
And whoever that person who claims to be the newest
a user of Linux, he gets to ask the first question.
And then everybody bandies about what the best resolution of that is till that's satisfied. And then they go to the next
least experienced user.
So, but don't think it's just for the least experience. It's having people in there who have some experience, which of course
give all these wonderful, wise comments and answers
And amazingly, even if you consider yourself an experienced user
Going to that meeting, a lot of times you get involved in what seemed like a simple question
And you get a bunch of other experienced users who have different approaches to it and
that you actually can learn quite a bit even if you're already a
a few years, more than a few years into using Linux.
So that's next Tuesday and everybody's welcome.
Next, let's see, on December the 3rd, it's a thursday
That's our Slack meeting, St. Louis Area Computer Club. Again, there is not a fixed presentation, so
it pretty much is what people want to talk about, but there's always something to talk about. Slack, the Computer Club, is a little bit more hardware oriented. So it's more talking about, you know.
what's in your computer, how it works. Sometimes it's, oh, how do I get Linux to know what's going on with this piece of hardware?
But we don't limit it to Linux. Sometimes it's how do I get Microsoft to know what the hell this thing's doing?
So we'll even talk about Microsoft there.
So those are things coming up on our calendar.
Anybody else have anything they'd like to point out in the calendar or other announcements?
No.
Yeah, they typed in L and O and then it crashed.
Okay, Ed, an L and O, and then it crashed. Can we have a little more content?
Yep, that was the first thing you mentioned.
Oh, oh.
The connection of the…
connection of the openettes the first
The first thing they wanted to log in
So they had to type in login
But they typed L and O.
And when they went for the G,
Crushy, ruin crushy roomy.
It's nice to know nothing has changed in 55 years.
Exactly right.
I don't have to say blogging.
now you get the prompt.
Oh, yeah. Oh, my God.
Okay.
Any other comments, observations?
Okay, how about some good questions?
And it sounds like I'm stalling and I am because our speaker stepped out for a moment
And so we've got a couple minutes until he gets back in so
Oh, I got a good question for people.
I've been using GNOME for
15 years. And for some reason
They have seen fit to take away
shortcuts on the
you know, on the keys
you know shortcut keys
you know for example up until the last version of SUSE, there was a shortcut key for
move current window
to the workspace up or down.
Well, there isn't in the next version of SUSE.
And I'll be darned if I can figure out where they're defined
I'm assuming there's a file somewhere under .config
where those keys are or where they live and where the definitions live.
Is anybody…
Any experience with that? Does anybody have any experience with that?
I guess it was a real hard question. Yeah.
Yeah, no, you can. So if you go under the settings
There's a key…
It's not fair.
I don't have it loaded up, so I'm not going to know off the top of my head, but in the settings panel, there is keyboard shortcuts.
And you can customize
the keyboard shortcuts
And I specifically remember customizing the one that you're talking about so i think it
does exist and is possible.
Yeah, but the problem I have is number one
it's not in the list anymore.
So you can't pick it and customize it.
And number two, I've tried five or six different ways to add it in
the settings and it's not there. So what I want to do is it was a custom shortcut
under my existing system and it still works and I'm trying to find it
in the config file, so I can just copy it to the new system.
Does that make sense?
He said this was Nome, right?
Say again?
He said it was gnome.
Yes, you know, current version.
So I have G settings. I think that's the GNOME program.
It's a command line program, G settings?
And you can…
It could show you it's all different kinds of schemas. So, I mean, that's where I would search. I don't know if your specific problem is in there, but.
Well, I don't see any…
there's plenty of schemas
P's, values, list.
Well, yeah, you have to pick a schema and then look for the settings in that schema.
I think that's how it works. It's been a while since I played with it.
It's fun to play with because I was looking for a way to do what you just said.
Which is to pull all my settings and then move them to another machine or something.
Okay.
uh…
Are you sure they didn't just change it to left and right?
Instead of up and down.
left and right are still there.
Right. I think they just…
But up and down are missing.
Right. I think that's what I remember. They switched from left and right to up and down to be more in line with the
trackpad gestures, I think.
Well, but there was no switching. You used to have four.
Up, down, left, right.
but they took away up and down.
Yeah, but can you even do a workspace up and down now? I thought it was left and right.
I've been doing it for 10 years.
And that's…
Right, but on the version where they took away the keyboard shortcut, did they also take away the vertical workspaces? Aren't they horizontal now?
No, you can make a vertical.
No.
Oh, you can make them vertical. And when you make them vertical, do the left and right ones automatically go vertical?
No, they don't work.
you know they don't work because there's nothing to the left and right
See, the basic problem is if you have multiple monitors
left and right aren't that useful.
you need up and down.
If you think about it.
Yes.
Because having left and right when you have four monitors sitting next to each other.
Yeah, but that doesn't give you multiple workspaces. For multiple workspaces, then you need to go up and down.
I mean, I guess. I thought it was just more of an animation thing. Does the animation go up or down or left or right? Because the functionality is the same.
like the functionality of a workspace is the same. It's just kind of does the animation go left or right or up or down?
Well, and it used to be you had those four options in the settings.
But now the up and down are missing.
So how do you make it go up and down? You said you could make it go up and down.
How do you do that?
I'm saying I have an older version.
where it works.
Oh.
And I want to find that those key settings in
the gnome settings
So that I can just copy them over to the new version.
The problem was a new version I did a fresh install.
On the old version, I've done upgrades.
So I still have it now.
I guess I'd have to try to grip for it or whatever.
Because…
yeah yeah no gnome uses that G settings thing, which is basically like the registry for Linux kind of thing.
Yeah.
Have you asked on the GNOME mailing list?
No, I haven't tried.
You might check their archives if you don't find the answer.
Try subscribing and asking.
Okay.
All right, Steve, you ready to go?
I have a gnome question.
Yeah.
So I'm getting frustrated. I think I'm about to retreat back to Windows on my desktop because
I just can't get gnome
And NVIDIA to play nice together. So I have an Nvidia graphic card that I want to use.
And I'd like to use the GNOME desktop because I just, I don't know, I don't like KDE that much.
But I can get it to work.
on Ubuntu and on fedora
But then when I sleep the system
And try to wake it back up, it will not wake back up.
At all. It's just black screen
And I don't even really have a good idea how to troubleshoot it because the only like
even using like control alt f1 and 2 in those to get the different
terminals like that's also
When you say sleep, what do you mean?
Sleep. Well, yeah.
hibernate or hibernate to RAM or what?
No, no, not to RAM.
Just, I think it's called S3 sleep. I might be wrong about that.
Boy, there are two ways. You can hibernate for ram
where the machine uses enough power to keep the memory set that the CPU basically stops
counting clock steps.
Or you can write memory onto the disk drive
And then restart the machine where the kernel restarts and then reads its memory configuration footprint.
off of the swap space and then goes back in.
If you're using
hibernate to ram
Because all you're really doing is stopping the instruction counter.
the CPU just goes into
you know no process mode until the switch is turned back on
If that behaves differently than hibernate.
you've got a problem with reinitializing the hardware.
And the hardware isn't.
getting set up properly.
So I would try
If one is blowing up
Try the other. See if you can get it.
We just hibernate to RAM and then restart. If that works and hibernate, the disk doesn't.
then you've got a problem initializing the hardware and you know where you're looking.
Okay.
I'll give that a shot.
The other thing is just avoid GNOME. I mean, it's a complicated mess that's
guaranteed to screw you up anyway.
Yeah, well, I really, you know, I know Waylon. I really like Wayland. I just like the
You know, I'm one of these people that can
you know, screen tearing just really annoys me and like uh
Hmm.
I like having consistent frame rates and stuff. I guess I've just been playing video games, computer games too long.
Yeah.
But so I really like Wayland.
And this only happens
Use FPWM. It has almost no overhead.
Yeah, this only happens under Wayland. It doesn't seem to affect X at all.
Which is making me
Well, but then again, if you talk to the people at Wayland, there might be something somebody's seen before
about initializing
the graphics system.
Yeah. Yeah. And what is…
But again, if you're coming back from
a right to disc and waking up.
A common problem with that
is that the
During the transition.
from command line to command line
display driver
the display driver libraries may be doing something to reinitialize some of the hardware.
and you find it, you know, if you do a hibernate to disk
then because you're writing all the memory out
and then just
cutting the power to the entire motherboard.
Everything, when you restart, you reboot the kernel and then it reloads memory
If there's any step that was done to initialize or configure any of the hardware.
that you did in your normal life before hibernating
it's not going to be done again that way when you reload.
And you can have a
you have a mismatch um
If your normal approach is to boot the system into the command line and then start X,
to get your window manager up, at least you've got a chance to see what it's doing when it starts up.
If you use a graphical login, you're fried because you can't see what's going on under the hood. All you get is the graphical system.
still be able to hit continue
But try just using Hibernate to RAM.
If you need to recompile the kernel or something and
see if that works.
If it does, then you know that the problem is that the sum of your graphics are some hardware somewhere.
is not being reinitialized properly with the
the reboot and then just directly reloading the memory.
Okay.
Yeah, I'm pretty sure Hibernate to RAM is what it does now because it's certainly not doing a full read when I turn it back on. It's like, you know, it tries to come back.
Okay, then it just pops right back up. Okay, then you're doing a hibernate ramp.
Right. And I feel like a hibernate disk, isn't that really slow? I think I'd rather use Windows than deal with that, it sounds like, because wouldn't that be super slow to load like
32 gigs of RAM from the disk.
No, no, you should be, if you use Hibernate for ram
then all the hardware is being left initialized and it just goes pop and it comes right back up.
Right.
But, you know.
Genome is a complicated mess.
So is KDE for that matter.
There are other window managers that if you like high frame rates and you want simplicity.
FEWM is one of them that is the one I use.
And it's a minimal window manager. It's not tiling. It just puts everything where it belongs.
So you might find that that gives you
better frame rates, better throughput
It doesn't do as there aren't there aren't
trays at the bottom and trays at the top and trays on the left and trays on the right and workspaces that pop up everywhere. Basically, you have a screen
you got multiple desktops and you can start things in them.
Period. But at that point, that's really all you need
And avoiding all the other crap
makes it a hell of a lot easier to get this running.
Another cute approach is avoid, you can avoid gnome
And…
just use like PWM. You use the
the brain dead simplest window manager you can just once. Just go into your
X and it rc
Comment out GNOME and put in TWM. I mean.
you're not going to use it day in and day out.
Watch that startup.
Open up an X term, a real standard X term, not anything else
and do hibernate for RAM, just run it.
And then restart the machine.
And see if that restarts. If it does.
then you know it's something in GNOME.
And you can go over to the GNOME side and try and figure out what the hell is going on.
Because right now, you don't know what layer is frying it.
Well, I know it's known because KDE works.
Okay.
Caitlin works and no on X works.
It's Waylon, it's Gnome on Wayland that fries.
Yeah, that's specifically no moment.
that I would go back to
XR kayak.
either read it or read it
I want to go back to that card.
Well, that's…
I like not having screen. I don't want to go back to the next word.
No, I'm saying use it.
Well, but if you go to the people that do Gilm.
And…
either Reddit or one of the other lists and say, hey, this is my problem. KDE doesn't try me and GNOME does.
Does anyone know of a problem with Hibernate and Wayland?
And X11 does not fry me.
So it's something about Wayland.
Mm-hmm.
how long you know and you'll find somebody else has dealt with it.
Thank you.
Yeah. Yeah. When I Googled it what you know specifically like you're talking about what I've
What they've come up with is some kind of
suspend where it saves the
Mm-hmm.
all of the VRAM, the video card RAM temp fs
And then it's supposed to reload that. And that worked under NYXOS, but I'm kind of tired of using NixOS now, and I want to use something else. And so
But that worked under Nyx OS, saving all of the VRAM to the temp.
Why wouldn't they put it in swap?
with the rest of the memory.
I don't know. Well, I don't have that big of a spot.
Well, I guess it's a hybrid HRM, you don't care. I mean, that's not going to work if you do a hibernated disc. But for a hibernate to RAM, yeah, that's safe enough.
Yeah.
But like I said, your problem is likely to be
that when you're recovering from that hibernate
They may be leaving some piece of hardware
that isn't fully initialized.
Okay.
And that may just be a difference in the way
Wayland's hooks for restarting are structured, then they may be different than X11s.
Yeah.
I mean, if you want to.
And I keep Googling it. I actually want to try the Hibernate 2 disk thing
Just to see how slow it is.
And see if that behaves differently.
It's…
Yeah, if it's not too slow, it might be okay.
Well, you're not going to use it every time. Just do it once to see what the hell it says. One of the interesting things about the, if you do the hibernate to disk.
Well, do you use a graphical login or you log into the command line?
Graph call.
Okay, that was your first big fuck up.
Log into the goddamn command line.
I don't know how to start. How do you start whaling? Is there a start Wayland command or is it only…
Don't log in to the command line.
And then run start X. And what's important about that
is when you start the machine up
don't run it in quiet mode.
And you'll see all the little lines that everything prints out when it's frying itself. And you can see the error messages.
Is there…
If you start the whole thing up in graphical mode.
then all you see is this pretty screen that doesn't tell you when the machine is screaming for help.
If you can't figure out how to type Startex, then do the simple thing.
Just temporarily do it.
Just for a short period of time while you're debugging this.
go into the system and have it start up
at the command line and you'll get just a command line
non-graphical, just that you get a text window
And you'll log in with your username and password and you'll type the word start X and hit return or whatever the hell Waylon uses to start up.
Yeah, I was just Googling that.
And at that point.
At that point, the important thing is, is you can type whatever it is
two greater than ampersand one
and type that to var temp
you know, startup, that out and hit return
If there's an error message in it, you'll see it.
Right now, all those errors are going out in the bit bucket. You can't find out what's wrong.
Wouldn't there be logs for like light DM or something
Okay, and where the hell are they?
Well, I'm sorry.
Off the top of your head, everyone here, tell me where those logs are right now.
I would look in the journal D first.
There.
And the answer is nobody effing knows because nobody knows how anything is set up.
is everyone's every distribution, every person, everything sets it up differently. If you're stocked using
System D, it's all buried somewhere in the guts of journal CTL. And if you happen to know exactly the right magic for recovering it, maybe you'll see it.
If you're lucky and you're still using syslog.
then it probably is in var log messages if you have everything everywhere configured to go to var log messages, but then everything's in var log messages and you've got to discombobulate it from 85 other streams of errors.
Unless you feel like using Splunk to mine the files, you're going to go nuts.
This way, if you just do start X or whatever the Wayland startup is.
Redirect standard error to standard out and pipe it to one file. And you've got exactly one file that has exactly what happened when exactly this thing started up.
Period.
And now you can do less on that file
And you'll see if there's an error message.
Yeah, I found that i found that.
As a matter of fact, if you can run this, you can usually
At least with X11, I'm sure Wayland is the same. You can start Wayland up in debug or verbose or something like that mode.
where it prints out everything it's seeing and everything it's doing. And obviously you're not going to do this every day of your life.
but wants to figure out what the Hades is going on
Just do that.
At that point you can…
Yeah, I found the Whalen equivalent of Star Dex to put in the chat.
But, you know, the
I spent…
a lot of years supporting Unix for EDS and Hewlett Packard.
And I learned the hard way it's easier to see what's going on.
99.9% of the time, you'll ignore it.
But you don't care. You're ignoring it.
But the time that you need it, it'll be there.
And that's why to this day, I start my machines up at the command line and I type start X. It's one word.
I get a collection.
And then if I want to see exactly what's going on.
I can look at the x and it r c
And I can run the thing in verbose mode. I can put whatever error messages I want out to the screen, whatever happens.
It's a shell script. The exit RC is a shell script.
So you can actually have the individual commands run them to greater than ampersand one and pipe them to something. Or you can actually run it in verbosa
the equivalent of a show minus VX mode, where you can see every line and you can just print out the return value of every statement as it runs.
And, you know, again, 99.99% of the time, you don't care because it all just works.
that the one time it freaks
you've got the log and they go, oh, that's where it blew up. And then you know exactly which command fried
And you know exactly what error message it admitted.
And then you can put the error message on and say, gee, this is my startup script.
This command gave me this funny error. Does anyone know what this means? And probably you'll get an answer.
Or you can check the duck for it.
Okay. Speaking of Gideon…
But the crummy…
if I could suggest, maybe we can
suspend this if there are any more questions.
And move on to our main topic of the night. Otherwise, our speaker will be all upset that we've delayed him. Oh, wait a second.
Steven, who's been giving the
I'm here.
Excellent.
I'm the one that's been wasting everyone's time.
However, there may be some participants in the audience who came specifically
who want to sleep tonight? Okay, okay, okay.
Um.
So let me turn it over.
Well, anyway.
I'm Mark from Workhorse Computing. Stephen's been a frequent speaker to us before. He did a
Fantastic talk about some command line stuff
Thank you.
last month at our wednesday meeting
And so tonight he's going to
Thank you.
talk to us about a voice.
give you more crap about the same basic topic.
That's right.
So yeah, this is the whole BSDM.
subject um
And this is about dealing with your command line. How many people here actually write
show scripts that are more complicated than
you know three lines or anything.
20 or 25.
How many people here are alive? Everyone who's alive and awake say, hello, Steve.
Hello, Steve. But that means they got to find her on mute.
So I got one person listening.
No, it means everybody else is muted like we like
It takes us a while to get our commute.
Oh, unmute yourselves, people. Let's have fun. This isn't the fifth grade. I'm not going to wrap your knuckles if someone asks a question.
I don't have a little ruler. I'm not going to beat you up anyway
Quite.
This answers the question, can we interrupt Steve during his talk for questions and comments?
Yes. If you have a question, feel free.
But this is for this specific issue is
partly about dealing with arguments to your programs.
And partly just some general introductions dealing with variables a bit more in Bash.
So command lines, when you put them in
There are a lot of ways over time that people have had for processing the command lines. I mean, going back to
Unix on PDPs in the 1970s.
Most of the approaches everyone had were terrible.
And everyone was carrying their hair out. So finally, someone came up with a thing called Get Opt.
And there was get up. Oops, I screwed up. That should be get up three
Get Opt is the C function.
that takes your argv
And hands back a variable sized array of all the answers that you care about.
So Bash uses get up from the command line. I'll look at GitOpt, the while loops
in the built of the bash cases
which are how you handle the individual statements
Bash has arrays. They're numerically indexed. The standard ones we're used to. There's associative.
dictionaries, maps, hashes, whatever you want to call them.
And how to set up functions, the basics of a function.
Ian Bash. And, you know, one more BSDM joke
Idle up nicely so
Bashtor's arguments, has anyone ever played with the command line in Bash?
Yep.
played with.
No. Okay. Okay. So there's one person who has the rest of you um
I hope we're awake. So the notion is we've got options.
And originally, all of the options were short. They were a dash and a letter or a bunch of letters.
or a bunch of letters and arguments. He never really knew.
And then you have double dashes.
Some of the switches on the command line may have arguments, some might not.
And then you're stuck dealing with them wherever other types. So if you look at here is an executable called foo.
Fletcher's floor, dim.
Is my path an argument to it or is it an argument to the program? And then you have DBAM and V1 and X.
There's no guarantee that people type any of this in any particular order.
There's no guarantee that if board is typed, it's not all in uppercase by mistake because someone misunderstood you.
There's all kinds of ways people can screw this up.
So…
The basic idea of the options is they're the things with the dashes.
anything with
dashes in front of it.
is a switch to the program.
Then we have arguments.
Borg has got an equal sign.
So it's got to be the argument to Fletch.
Minus D has BAM as an argument.
we think minus V1, yeah, one's probably, but notice there's no space
So it's up to you to know whether it's minus V with a one argument or it's the
Minus V1 switch
No idea. And my past might be the argument to BIM and it might not.
guessing the program arguments might be my path and bam and one
And again, they might not.
you can see where every scheme that people have for dealing with command lines is messy.
GitOps was the first version of this that ever worked.
It takes in a list of possible switches
it takes in
some metadata that tells you whether the switches have arguments
whether they're Boolean.
whether you save the same switch more than once, it ups a number like you have minus Vs. Everyone's here seen
Minus V is verbose. Minus VV is even more verbose. And minus VVV.
Minus V, minus V, minus Z is really verbose.
And so it doesn't set the switch three times
It says, oh, V is incremental.
As opposed to just Boolean.
So all that metadata is about
knowing how to deal with each of the individual switches and what it's supposed to have so that you can
you can actually compile
that an input line into real arguments.
And the whole point of that is I've got to take all the switches and all the options to the program out
Because what's left are the actual program arguments
like an input path. So if you do a DDIF OF something.
input file, output file
there might be an argument at the end of it.
or
tar minus something something and you give it a directory name. You'd have to know the directory
is being processed by the program as a whole. It's not an option that changes the way the program's running.
So…
One thing that GetUp Long added
was in instead of having just a dash and a letter, there were double dashes with long names
And the great thing about that is you can say foo equals bar. So you've got something that says, oh.
I know that bar is an argument to foo. End of story.
There's no guessing. You don't have to figure it out you know
Every language since then, and that was in the 80s
has used some kind of get up long.
Pearl calls it get up long.
bash just uses get up that handles long switches. Python's got something similar. Every language today
uses something get up-ish
And gives you the same basic life cycle for these.
Now, Bash stores the arguments in a ray
whatever order they were typed is what order you get them in. So it just takes the command line and breaks it apart on white spaces respecting quotes.
Now, a quick note when we go into this. There's a dollar splat and a dollar at sign.
If you don't put them inside of double quotes, they behave identically. They give you a spring.
of everything that was typed.
if you quote things, dollars splat
there's a variable called IFS.
that is the
It's the variable you set that tells bash how to split arguments apart.
And it normally starts out as white space. It's a space
Followed by a tab followed by a new line.
And what it'll do is take the first character of that IFS variable
And it'll paste all the arguments together with that.
So that was what you
what Bash thought you typed in.
Dollar at is kind of neat. It gives you all of the words separated.
So that allows you to pass the arguments into a function that processes them
keeping each word separate so the function can do what it needs to.
Does that distinction make sense?
Everyone with a heartbeat, just say hello.
I'm still thinking about it, but I think it does.
Yes, it makes sense.
So, well, if you look about it
Dollars splat inside of quotes is how you print out the argument list.
And I'll show you an example of that with a
you know say you want to create something, a function that prints out log messages
It'll take all of its arguments and prefix it with a date
and a program name and a tab and then dollar splat. So it just takes all its arguments and just
tax them together inside of the one long string.
Dollar at sign is how you iterate down the arguments and see what each one of them look like. And it's what we'll be using
the process, the command line.
Now, an interesting thing
I was thinking of…
Hmm.
Why am I thinking of that as exactly the reverse because you're saying
go back to that previous slide.
Dollar splat takes
Okay. Yeah.
You're…
And it takes all of the arguments and packs them together with a space in between into one long string.
No.
dollar at sign
Put a…
dollars flat produces a single scalar string
dollar at produces an array.
Okay, I misunderstood you then. I thought it was exactly the opposite.
In Pearl Speak, dollar splat is joined with a space
at underscore.
Okay, I thought…
dollar ad is just a flat array of things. It's an array of separate things by space, you know.
broken apart by spaces.
Okay.
So you can think of dollar splat as pre-parsing and dollar add as post parsing.
Okay.
from the command line point of view.
All right.
Did that help the way I said it?
Uh-huh.
Okay. So an interesting thing is
They're dollars flat and dollar at
are there's one of those in the main portion of your program
And then every subroutine, every time you call it has its own local dollar splat and dollar at.
So everywhere you call it, because it's localized.
you have, if you're going to write a function
that processes the command line.
you'll have to pass it as dollar app.
Because then the subroutine gets each of the arguments by itself. You'll see how that works in a minute.
But a lot of people screw that up they try they
you forget to pass the arguments in and you drive yourself crazy. That's why I put this up front.
So there is a function, it's a shell function called get opt
It is different than the built-in function get ups.
world that is in Bash.
The get ops is much more simplistic.
I'm going to ignore it from now on.
get opt the shell command.
is what you want to use.
So again, there are two types of arguments. If you look here, you'll see a minus V1, minus VAM minus X.
These are short. They are a single letter.
that might or might not have an argument.
the argument might be glommed up against the variable.
or excuse me, against the option
Or it might not. So these two lines are identical.
Oops, except for the typo minus X I left off.
Minus VDX1 BAM is the same thing.
Whatever of these needs an argument, it will consume the number one
The next one that needs an argument will consume the BAM.
Can anyone here imagine that that is confusing?
Yeah.
Because if you don't know whether DRX takes an argument, you don't know which one of those is going to get one and which one's going to get bam.
Mm-hmm.
So the old single letter dash approach
is horrid. Don't use it.
Now, the long arguments are a much better way to design it.
You can say blech equals board or blech flirt
If you're writing shellcode
And you invariably use long options with equal signs.
Whoever has to maintain the code, and that poor swab may be you.
is always going to be happier.
Because the long options
They give you some idea what they mean.
you know if you give a minus g7
what the hell is G for?
If it's minus go to hell seven and you know, whatever hell you're going to, bang, you're done with it.
So you can't stack them. You don't have minus fletch, minus minus foo, and then blurt and something it's just minus blush equals blort. They have to be attached.
This is a much better way of passing things when you write code, even at the command line.
it makes more sense.
Use long arguments is the first thing I can tell you. Always write stuff.
So that it takes long arguments for anything that's non-trivial.
The short options
can be kind of nice if you've got debug or verbose.
you know, switches that are binary
Minus D for debug, V for verbose is a classic, or minus H for help. I should have included that.
Minus, minus help and minus H both get you the help message, fine.
But the minute you start putting arguments into short options
it just gets too hard to parse.
So let's look at the
The first way of cleaning things up
What you can do is pass everything you're doing to get up
You give it a list of short options with the minus O.
You give it a list of long options and you can have minus short, minus long, but they don't fit on the screen.
And then…
Double dashes in shell land say these are the end of your options.
Whatever follows are the arguments to the program.
So what this line is saying is that get op is being told minus O minus L.
with its arguments.
And everything else
are the arguments to the program that's running
Don't fiddle with them.
Now, that dollar at
is whatever the person typed at the command line.
So set.
The cute thing is the
The set function built into bash
sets a variable.
By putting set dash dash
you're basically saying
Set the variable that has no name.
And the unnamed variable is the arguments.
So this jam loads dollar splat dollar at
with whatever the result
of calling get opt is.
By evaling that.
I get this is a long list of
whatever. And that's put into a new set of arguments.
What GitOps does is take whatever mess
Somebody typed in for you and it puts it in a sane
order for processing
And pukes if there are any errors.
So this double dash
separates the get opt
switches from your command line. Does that make sense?
Yeah.
Because you might have a minus O in your program and get up's got to know which is which. So that double dash is an important thing. It works in Perl. It works in, I think, Python too.
Where if you're passing arguments to
pearl that are part of the program that's running, you can put a double dash and Pearl knows not to parse those. It just leaves them on the argument stack.
Set minus minus is how you reestablish
the arguments to the program.
This is an undocumented feature of Bash that you will find if you read the get opt
documentation carefully.
And then an eval is a bash built in
And this Pearl, Python, everything else has got one that says, take this text, interpolate all the text, and then reevaluate it one more time.
as a piece of code and keep going. So it allows you to recompile
So whatever the result of GetOpt is.
you're going to evaluate that text
by setting your arguments
to whatever that is.
So now what's that?
So what you start getting
you get this mess.
will be sorted. So now we can look at this and we can say, oh, Bletchensport
Tim has no arguments.
B takes a BAM argument, V1 minus x, oh.
And my path is the one argument to my program
separated by a double dash.
What's important about this
is if you walk down this
Every single one of the options
is separated out
If someone gave you minus VX1, you're going to get minus V1 and then X.
So all of the options and their arguments are paired.
Any argument that any option that doesn't take arguments has
a dash following it.
So, you know, minus X doesn't have arguments. So the next thing there is a dash.
Minus D, bam, you know that if you're seeing a minus D, there's an argument.
You know that if we're seeing minus minus BIM,
The next thing after it has a minus because BIM doesn't take arguments.
Now it's really easy to walk down these one at a time
And say, oh.
I know what comes next. I saw a blesh. I'd better see an argument.
The very great thing is the minus minus is how you separate out
switches
which control the program running
from arguments that the program consumes.
Did that make sense the way I said it?
Ooh, say that again, please.
If you look on the left side of this, there's all the switches that control how the program behaves.
this might be verbose this might be
a working directory. Whatever these are
These are consumed by the program to tell you how it operates.
This is the argument to it. Think of tar.
If you give it a tar and you want it to compress, you do a minus, I think, X, right? Or minus
G or whatever.
TAR minus g fubar.
Minus Jesus, I'm telling her how to behave.
So it's going to consume all of this stuff
before it knows what to do with that, because these are the options that control the program
This is the input that the program consumes.
Do you see the distinction I'm making?
Yeah, yeah.
I'll show you in a minute, but your command line loop
is going to eat all of this
Strip off the minus minus and leave this stuff.
Because that's the main part of what your program operates on. And I've got a good example is if we get through all of this, I've got an example of something I wrote today
that compiles a copy of Perl and uses this, and I'll show you what it means.
where your cursor just was, the double dash right in front of my path
Right.
Is it my eyes or is that in red instead of in black?
Yeah, it's in red because I was talking about it.
Okay. All right.
Back here, it was in black because I wasn't talking about it.
Gotcha. Cool.
It's called highlighting.
What a concept. Damn.
So your command line loop is going to walk up this
until it sees the double dash now
It's the first thing you see is a double dash.
There are no switches. This is what makes it so easy to process.
Because you're going to walk down this
The facts that get ops didn't puke means that all these are valid.
So if you can consume all of these
And then stop at the minus minus
your life is good because whatever comes after it isn't your isn't your
options processor. It's stuff you're actually consuming.
So…
One thing we can do
Just to make this line a little bit readable.
I can say, here are my short options.
Now, you see this colon
That says that this takes an argument and this takes an argument.
So D colon, V colon X,
says I have two short switches to take arguments and X that does not.
The long arguments say that I have, oops, I screwed up. Those minuses shouldn't have been there.
downside of writing this on coffee.
So you got Blort and BIM. I'll clean this up and give it back to
Gary, Bork takes an argument and Bim takes no argument.
So if you see BIM, you know there's nothing following it.
And there's versions of this for optional arguments and other things.
you can see that the argument has to be an integer. You can say it has to be whatever.
For the moment, it's just there.
Now, if you had to read this.
and look at it.
My arguments are whatever I get back
And I can set from the arguments.
It doesn't look that different besides that short and long are maybe a little bit more descriptive
But where it gets
Oops, I apologize.
Wrong.
Oh, come on.
Oh, here we go.
So the key thing here is
by storing these arguments.
it makes it much easier
to have a better error message.
Because I can say if this
comes out, if those arguments
match the word unrecognized
This is a regular expression match in bash.
then I can print out a usage message, call my usage function and say.
you gave me bad options. These are the ones you gave me.
And this message will say bad list of options
you know minus foo unrecognized, and it'll tell you what the bad option was
By storing this and then outputting it, I can give people a usable readable error message.
Does that make sense?
Yeah.
Okay. This means a lot of work because now you don't have to walk down each of these and say, do I recognize it? Oh, issue a memory message for what I didn't recognize.
It's all done for you. Your error message for bad command line options
is just print out what you got back if it says unrecognized.
And there are a couple of other things it can give you. If you say it has to be an integer and the argument to it's bad, there are a few things you can actually search for here.
This is easy to see. Now, if I get past this, I set the arguments and keep going.
So now I've got these arguments set.
What do I deal with them?
So I can…
iterate my new dollar at sign, which now is in a processible order.
When I see the minus minus.
In it, I know to stop. I've dealt with the switches
I'm now down into the program arguments.
So an example of that, if I eval the set
The while colon semicolon, has anyone seen that before?
No.
Is it while true?
Yeah, it's the moral equivalent, but instead of re-evaluating true
And remember, bash is not compiled it is
It is a translator.
Perl and Python are compiled languages. They don't compile to assembly.
They compiled a bytecode.
But…
Bash actually has to figure out what each line means every time it sees the line.
So the while colon semicolon is a speed up it says
don't evaluate any further, you know to keep going. It's the equivalent of C of before semi-semi.
four within, you know, a for loop in in C with an empty set of arguments
In Reiku, there's a function called loop
It takes no argument. The only argument it takes is the closure of what it's going to run.
I read that as due forever.
So…
Yeah, that's, yeah, that's wild forever without bothering to check. It just saves time.
It also tells the person reading this
oh, this is going to go forever unless you do something to break out of it.
So it's a quick warning. If you see that.
Expect a break somewhere in the code or an exit or something.
One meal?
So what's wrong?
Ready?
I thought there might have actually been a question, sorry. So case dollar one
There was…
dollar one dollar two dollar three are the elements of dollar at.
So these are the arguments in order. And again, these are
These were reestablished when I did this set minus minus.
The dollar one is the first thing that get opt
sorted out and put in the line.
We're going to…
And if dollar one, if I look, if it's a double dash.
Then…
I shifted off because I'm not going to use it anymore.
And I break because the program arguments are all that's left. I'm done processing options.
Did that make sense?
Yes.
Okay. Well, then…
Now, one thing I know about blech
Fletch does take an argument.
So if I see Bletch here in dollar one
Yes, if you had to, where the argument to Bletch would be.
How about dollar two?
Oh. Uh huh.
Now, I'm going to shift something off the stack as I have to consume two things.
And I do the rest of this for minus minus help, minus H minus X minus D.
Anything that I know takes an argument, I take dollar two
shift it. And then every time I go through this loop, I have to shift at least one thing off.
Do you see why?
Yes.
So every time through this, I'm consuming the first item.
If the item has an argument.
I may have to consume two things.
So there's an extra shift in here. But otherwise, there's a shift.
Oh, okay.
And this will just chew through. It'll say, oh, is the first one blech?
Fine. I know I'm processing blotch.
Its argument is whatever follows it on the stack
shift that off and then shift off the bl.
So here's two shifts if I have an argument.
Dash doesn't have any arguments, so there's no extra shift.
And this will chew through. If you have something like minus H, it might just print the usage message, which doesn't exit.
So it might just say usage and double semi because it doesn't need to do anything else. It's going to exit right after that.
And again, if we can get through this, I'll
I'll show you something I had to write today. I've written this 14 times and every time I write it before now, I was working for a place
where I couldn't take any of the code with me. So I've gotten very good at rewriting this.
Now, okay, the bad thing about this, if you look at it, blatch for boast, debug.
that blush equals dollar two
means that I've got a global variable named blat.
Well, that's easy to process. It's easy to set in here.
But now I'm starting to get global variables and people write something else. And if this were verbose, maybe someone else said verbose to something else. And now I don't know how it's that. It's just easy to step on yourself.
So another way to look at this
is I can use a subroutine
to come to
compartmentalize the variable space.
There's a typeset function. It's built into Bash.
it's in an it's
Another synonym for it is declare.
And time cent minus R says, make this read only so that if you accidentally step on it, you know.
So typeset minus are sure to equal this.
And one thing I learned, by the way, because you've always got a
The variables and the arguments in bash
can't have any white space. You can't put short space equals. This will confuse it.
So a neat way to make your code a little bit more readable is instead of lining up the variables and having this jagged line of equal signs with stuff on either side.
If you line up the equal signs, it's a lot easier to walk down it
and see what the variable names and the stuff on the right side are.
So, and again, I screwed this up here. I'll fix that before I
upload a real copy to Gary.
But the short and long, if I accidentally try to modify them, I will get a runtime error.
So then I can say local argv equals this
And the rest of this.
All of these that are typeset. Anything I declare with local
is only has meaning inside of this function.
Did that make sense?
Hello?
Yeah, you're cooking variance.
If I put everyone to sleep this quickly.
No, you haven't.
that makes open.
Okay.
Just in the process what you said.
But it's like a lexical variables in Perl, my variables
Python finally has the same thing. You don't have to have everything be global.
you can define a variable as only living inside a function.
So that's what we've done here.
the local
says here is a variable that is local to my subroutine
Typeset does the same thing it it
It localizes a value to the context in which that variable is declared.
R marks them as read-only. And that just saves you from typos.
If you accidentally assign something, it just saves you.
And again, dollar at, oh, that should have been dollar splat too, sorry.
are they're always local
to whatever context they're used in.
So you never have to wood now
Everyone knows what this means, right?
Oh, come on, people.
Nope.
This is the motto of Pearl.
There's more than one way to do it.
Oh, yeah.
obtuse in the first.
Tim Quitty, there's always more than one way to do something.
So I can have…
this function command line here
And now I want to export these variables. If I make everything local to the command line function.
I really haven't, you know, it's not very useful because I got to get something out of it. I don't know what somebody typed.
So one way to do that
So…
is I can typeset the variables
in the global space.
And then assign them down here.
If you do a tight fit minus L,
That says that anything stored in this variable
will be lowercased.
And if you have, let's say, Bletch's true or false.
How many ways are there to mistype true?
Caps lock. Is true capital T true or is it lowercase T true or is it true
typoed or is it true all uppercase or did I accidentally hit the shift key at the end of it? Who knows?
If I declare a variable lowercase.
All I will ever see is the lowercase version of it.
For things that are effectively enums.
This makes life so much easier because people can't type it wrong.
Yeah, that makes sense?
Yep.
The other cute thing about this is if I typeset it up here.
Anyone walking up to this
knows that there's a global variable named flash.
And they can look at that shit that has faces around it. I was thinking a pearl.
God, I really typed a lot of this.
So, well, that's equal to, I can look for Bletch equal sign
And find it and be done.
I know that I'm looking for it. And if there are eight of these
At the top, I know what in my code I'm looking for.
Now, the trick there, another way to do this is I can just hand everything back
And I can say my arguments are
Execute the command line or whatever it hands back, those are my things. The problem there is.
If you push these back as a list.
You have to know what order they're in, an argv one is this, an argv is that.
it becomes kind of messy accessing it.
This works. People used it for a long time. But there's a better way than having to memorize the order.
you can have an associate of array.
And they call them map, call it a dictionary, call it a hash. There's different names in every language for these.
But it's an array indexed by name, by string, rather than by number.
So instead of saying lord equals $1 one or two
I can say argvort.
Now, what I've done is I've declared a variable
that has blort as a key, and I'm setting it to a value.
Bingo. Now I can look at this and I've got all of them in one place.
There aren't 15 variables scattered all over.
And I can tell what they mean.
So if RV is verbose is turned on, I can say argv is verbose equals one. I just arbitrarily set it to one. Otherwise, I set it to an argument.
So now I do this. I can do a typeset minus a of argv.
And it looks nice because people look at this and they say, oh.
argv is an empty array. I wonder what you're filling it with. And pick your name for this arguments, program args, whatever.
And then I go through somebody, again, if I define this at the top, if I don't.
Simply assigning it is enough to create it as global and simply assigning it the first time
With the square braces and a word inside it.
is all it takes.
to declare the global variable and keep going.
So it depends on if you don't like that tight set, if it gets
confusing, keeping things, you can do it this way also
But now further down, someone can say if argv are verbose.
I can ask, am I running verbosely?
it's a bit more text down here when you're testing it, but it means you only have to keep track of one variable, not 45.
And it means that adding something new doesn't drive you crazy. You don't have to find the variable for it.
Um.
Thank you.
Another way to do it
is now I've done this, but I've got another problem
And I, oops.
Nobody's asked this yet but
What do I do with what's left in dollar rat? Those are the actual program arguments.
The easiest thing to do is hand them back as a list.
If I do that.
I can create a read-only variable called command args
that's whatever I get from reading the command line.
My argvs, or call them
options, ops maybe
is whatever had dashes on it
And these are all the arguments left
After processing them, it's whatever was after that double dash.
So if I'm writing something that does it
tar and I want to pass switches to it
I've read all the switches and now here's the arguments that I'm actually processing.
Does this make sense?
Yep.
Okay, so at least someone's, I haven't killed all you. Now, another thing is
And this is something I put in the top of everything I write.
I call base zero, endure zero.
I break up the program name.
The neat thing about that is neat about
When…
GetOpt gives you the nasty gram
for unknown switch.
If you give it a minus n and a word.
that will be, it will tell you the name of the program. So it says invalid arguments to
whatever it's minus n argument is.
So the cute thing here
is I can…
by putting this base name in.
I can get a more reasonable error so that if this shell script is called from another shell script.
you know, 14 levels deep into my operating
it will tell me which of the
the pieces of code actually issued the error message.
And this can go a long way, especially if you've got something running in the middle of the night and all of your logs are centralized with journal CTL or
or syslog, you'll get the actual name of the running program that complained about it.
Which makes it a lot easier. Hey, who here has a minus minus blanche argument? You got to
research it and find it and you go crazy
And what if three people have got it. Now you're all fighting over whose blech was the one that blew up. This way we know exactly which program died.
And that's as far as I got.
The basic things you can do, and this is completely reusable
If you have something that does command args equal
command line of this command line
One cute thing is
The way arrays work in bash
If you access an array variable without supplying it an offset.
You're accessing the zero entry.
the first one.
Which makes it really easy to ask the question
do I have any arguments?
I don't have to go, you know, curly brace this in a zero
If this is empty.
You forgot to give me any arguments.
it's so i i can i can call fatal, tell you
No arguments. He blew it.
Otherwise, maybe I validate a source file, I do whatever, and if
my idea verbose, then echo verbosely something.
This is a general structure that'll work for
anything you're ever going to write in Bash.
It's…
Okay.
Okay, I got the first part, but I don't necessarily understand this for the most part. What are we doing here?
Well, basically, if I store all my switches
If I store everything that you pass me
Maybe I should have called this ops. I'll rename it to that.
If these are the options that I processed on the command line and you passed in minus minus verbose, let's just say.
or minus V is verbose.
I said argv verbose in here.
It's global.
So now when i get
and someone wants to know whether or not verbose was set on the command line.
They look in the RGV and they say, was verbosecet?
If verbose is not empty.
Then I echo something.
or I say whatever I'm going to say if I'm running in verbose mode.
My command line arguments
Okay.
the things I'm consuming are handed back by this function.
all of its side effects that define how the program is going to run are left in RV.
Okay. Yeah.
And that separates it out pretty nicely. You know that the command line arguments are there and it's read-only. You can't screw them up.
Yeah.
And so I can look at command args as zero, command args of one, whatever.
Or I can walk down them and say, okay, now that you've given them to me, what do they mean?
But if I want to process one of my initial switches, that's in the argv hash
And I look up verbose in there, and if it's not empty, then I do this or i
I know…
I know what you gave me is the argument to that switch.
And if you want to see more about that one.
Man of Bash.
tells you everything there is to know about bash
GetOpt is a separate program. Again, there is a get opt
plural that's in bash
that does not do nearly as much work as this does, and it is not worth using.
Any system that has bash on it will have get optic expand line.
just use the get up command line utility.
it does a much better job.
And if you do a man three of dead opt that's the C version of this.
that goes back to the 19…
19…
early eight years or early 80s
late 80s, I think is when that came in.
I'm trying to remember when I started using dumbled ashes, but it was around the 80s.
It's been there a while.
But the get opt-in-C is where long options were first introduced.
19.
And you can see how it's supposed to work.
That was ridiculous.
late 80s, early 90s, it was around then.
And that is it.
What I can show you is an example of doing this.
Bingo is bingo.
Fond.
Too small.
bump it about four times.
Is this legible for people or do I need to raise the font size?
That's bigger four times.
Okay.
If you were running at all, you just do control plus.
Yeah, Noel, I…
But that doesn't work in my editor.
Thank you.
Hang on.
It does work at the shell level, however.
Drew.
Um.
Why don't you use a good editor?
um and i
That's the feedback.
This is the one I like.
I'm cheating.
The nice thing about Vile, Vile has a lot of advantages
Oh, it's a good editor.
But…
The main thing is, unlike Vim, it actually looks like BI.
But you'll see what I do here. I start the program and I type set
phase one, endure one. One thing I'll do in a lot of programs
Let me do that here. A lot of times it's handy to know what directory you're executing from.
So I'll do a tight set
clients are then be equal to cd dollar zero
PWD minus L for serving the SIM links.
And the…
Is there a chance you can make that one size larger?
No, I'm just going to read it here.
Another thing you can do is if you have the view
So here.
option button above that
Not Stephen, but the rest of you. If you have the view option button, you click on that.
Okay.
There's the option to fit the window or original size. If you go to original size.
It'll be hard to find his cursor on the screen
But you will get nice readable font.
Okay. Well, is this line this is
Boy, I see that.
Well, the only point is
If you CD to this directory.
you know that the point where you start
this is a valid path because you're running it. If you do a PWD, it gives you an absolute path.
So this is a quick and dirty way of
converting whatever path somebody used to start the program, dot, dot slash foo slash bar.
into just a single absolute pass. The minus L preserves any SIM links.
But then again, the date that I start at, my log file.
this regular expression are all read-only things that are globals.
What I generally do with globals like this is I'll make them read only so I don't accidentally step on them in the code.
even but you might
Yeah.
follow up on that request. And can you make that any larger?
Because it's hard to follow.
Um.
This is the large, well, let's see
How about that?
That helps.
Keep going.
Okay.
How blind are you people? How's that?
Perry. That's better. That'll work.
Lee's pretty far.
Okay.
So…
An example of using dollar splat.
I call the log message function.
It puts a new line. It tells seconds is a variable that tells me how long the current executable has been running.
So it just gives me a quick idea of how long
it's taken to do things.
Base zero gives me the name if there's a bunch of these running.
a tab and then dollar splat just nails the entire argument list together
with spaces and splatters it out. One greater than ampersand two puts it to standard error.
fatal logs
whatever you told me was fatal in front of it and then doesn't exit.
This is execute which is execute
runs something.
My main use of this
is running shell codec
If I assemble a line of shell, this will print it out so I can see it.
validate
If I pass this in with dollar one
If I pass a command and a bunch of arguments to it.
This will verify that at least the first thing in the string is executable.
So if I botch
a command line statement with a non-existent file
I'll find out specifically why it couldn't run.
It gives me a bit more detailed error messages.
When I'm trying to find out what blew up at two in the morning.
eval of dollar splat takes all of the arguments, gloms them together, and runs them as if they were code.
If they return non-zero.
I tell you what the dollar one was. I don't do the entire line because you just saw it a minute ago.
And then I tell you its exit status because that's all you really know.
Whatever it emitted.
is…
He's going to come out. Now, another approach to that is you take this
And you can say log message
of eval of this
And that will print all of the stuff out
But it just gets to be a lot of, this is a good place if you had a verbose switch
you could say something like, you know, if I'm running verbosely, log a message with the output of this otherwise
don't bother. Usage is just, you know, cat some things with an end.
Here's my command line processor so
I've got my longs, which don't have double dashes on them. I got to go through and clean up that presentation.
So I help and recycle help
Yes, what that does recycle saves you from recompiling the copy
of Pearl is if you're doing the installation phase and it blows up
you can reinstall things. V&H are for Bose and the short version of help.
So again, I do the set.
This is exactly…
what you saw before.
And then…
I always know there's going to be a double dash somewhere. It may be the first thing if someone didn't give me any command lines switch.
help and minus H mean dump the usage.
That's going to break out of this infinite loop because it
If you look up here the usage
does an exit.
So this gets me out of running the loop forever because the program terminates.
Otherwise.
I've consumed one thing off the stack.
Now, if one of these took an argument
I know I'm consuming two things off the stack because of its argument
Another way to do this is if you have an optional argument, by the way, you can say if dollar two
starts with a dash.
then there is no argument to this.
One of the great things about the way GetUp orders things
is you know
that if there's an optional argument.
The next thing, dollar two, is going to begin with a dash.
Period. Even if it's the first dash of double dash.
it's going to begin with a dash. Anything that does not have an argument will have a dash as dollar two.
Thank you.
So it's easy to tell with optional arguments because you can say if dollar two
begins with a dash, skip it, otherwise save the argument.
or use a default or something.
So now I've shifted all of this stuff off. I've consumed everything up to here and gotten rid of it.
The way this program works
you can get you have to give it
an input. So you can run it or you can optionally give it an input. You can give it the name of a tar ball.
Or you can give it the name of a directory that's a Git repository.
Or you can give it the output of the previous execution on towering the tar ball.
This is one of the program arguments.
Minus minus recycle is an option.
The options get consumed and they control how the program
deals with its own arguments.
So here's a case where I've got an option
and an argument.
Does that make sense?
Thanks.
Yes, but aren't you, are you expecting people to put two dashes between those?
No, that's the beauty of it.
Okay.
You don't put two dashes there.
But GetUp does.
So what I'm going to get back
The character string I get back from running get opt
Even if somebody puts base zero, path to purl star space minus minus recycle.
the string I'm going to get back is minus minus recycle
space dash dash
Pass to Pearl, whatever the shell expansion of path to pearl star is.
And that made me pearl-12345, whatever it is.
The shell does this expansion and then that becomes the argument to our program.
But me, they don't have to type double dashes.
get up inserts it when it runs here
So that I get a double dash and I know when the end of my switches are.
I can see the last option.
Because it ends with the double dash. If you type no options.
So if someone ran the program
This way.
if someone ran
the program with just its name, which assumes the current directory.
All I'm going to get back from calling get up is dash dash
If someone runs it this way.
I'm going to get dash dash
space has to tarball.tar.gz.
Does that make sense?
Yep.
Yeah.
Get up and search the dash dash for me.
So that I always know when the switches end.
Yeah.
And because it's dash dash, I always know
But if any of my switches, any option has an optional
Argument.
If the next thing isn't a dash.
Then…
I know that there was an argument. If the next thing is a dash.
then I know that they're
won't be a dash following me and I have to consume that as the argument. If the argument takes a leading dash.
Then GitOps puts it in quotes for me so that I know there's not a dash. I can tell the difference
between switch arguments that have dashes and program options that have dashes. It makes it much, much easier.
Because now the only thing it ever takes to process them is this.
I have a case
And each of my options is one
One example in the case.
And if there's a short and a long that do the same thing, you do it this way.
That's it. This is all of the logic it takes
this while loop
will deal with the output of get opt every time.
So it's much, much simpler.
And you know that if
somebody passed, if they typoed one of the options
If they typed recycle without the E on the end of it.
They're going to get a nasty gram up here.
So I only have to process
the options that are valid.
Because invalid options don't get this far.
They get removed up here.
Do that?
But how does it know if it's missing an argument? How does…
How does it know if it's missing a treatment or not?
Oh, the notation
Well, here, where was oh here.
Um.
Oops, hang on let me let me
get this back on the screen.
And I will show you.
get off a set of…
The metadata
Specify.
or get opt tells you whether the options are optional so here
Oh man, get up.
Um.
the offspring and the parameters which are parsed as parameters in the synopsis so
Um…
So along character options may especially like once
You separate them with commas
Each…
Each option name in long ops may be followed by one colon to indicate it has a required argument.
or by two colons to indicate it's optional.
Oh.
And here's where that name that gets used in the error messages is.
And the short options
are one character options.
And again, they can be followed by one colon or two colons
And you can use a plus or a minus to change some things but
In general, another thing you can do is because
Bash is the default bot.
CSH has a different format where you can use minus and plus
And they mean if you turn something off, you put a plus in front of it.
So CSH and tchsh
have slightly different semantics for the way you use supply
the program options
Yeah, unquoted, never, ever use it because white space, anything in
the actual switches
that turns out to be
important to the shell is going to fry your ability to run it.
But…
There's…
Now, this thing uses the actual get opt
C library. So what it does under the hood is classic GitOps 3 so if you
If you do a man three get up, you will see what all the different ways you can do things are.
So all this is really doing is it's a wrapper around the C function that passes things in.
If this thing pukes on its own parameters, if you give it three colons.
you're going to get an exit two.
If it blows up.
And testing, if you're in test mode, it'll exit with a four.
But in general.
in your own testing
you're going to get rid of this.
In the modern world
you don't usually get out of memory unless you're running on a Raspberry Pi and even those have gigabytes today.
So normally what you're going to get is an error one
Which means that someone put a bad switch in.
there's positively correct, there's get up
And that you can set that in the environment
Bye.
I'm so excited about this. I love the idea of tea.
Yeah.
This is basically you can put this with the parameters or you can not put it with the parameters.
The double dash is unnecessary because it does a good enough job of figuring out what's what.
The only time you need a double dash
is…
like I did here.
Thank you.
Because, okay, if this is processing
I say it right.
Because your stuff probably has dashes and double dashes in it.
This double dash separates out
The…
switches that are being supplied to GetOpt that control how it works
And the arguments to get opt, which is what it's parsing.
Does that distinction make sense?
So the minus O short and the minus L long are the metadata that it uses to figure out what these mean. That double dash separates out the information you're supplying to get opt
with the input that it's processing as work.
Does that explanation help?
Yeah.
I had a question about, so you
And the same
You put the dash in with the base zero there, but then your error message down below, you're putting base zero again.
Is that like duplicating it?
Well, because you may, yes.
belt and suspenders.
The other thing is.
you can have a problem
If this line is long.
it can get split
into multiple lines in a log file.
And this way I can always search for bogus space program name and find it
where this might have wrapped onto a new line and there might be
it might not be together.
So yeah, I could have left this out.
I could have called this bogus command line.
There's a lot of things I do that are belt suspenders because I've been there at four in the morning banging my head against log files that didn't have enough data in them.
But yeah, you're correct. That one is actually extraneous.
And…
So now, if you look at this.
I've chewed through dollar at sign
I found the double dash and shifted it off.
So now…
This one…
If you look at this point in the code, the dollar one here.
is going to be the path to a tar ball
or a directory
a repo or a previous Antar directory.
If someone runs it this way.
the return of GitOps will be dash dash as a single string and nothing else.
If someone runs it this way or this way or that way or that way.
I'm going to get dash dash space dash
Whatever they typed has to get repo.
And I'll know that there are no more switches.
So that loop is going to run exactly once and consume the double dash.
If someone runs it this way.
they're going to have the program
Dollar one is going to be recycled. It's going to be followed by dash dash
And nothing.
And in this case, I'm going to get
Recycle space dash dash space
And the next argument is he passed the pearl. And when I set that, I'm going to get recycle is $1.
Dash dash is dollar two.
in the past, whatever this extrapolates to is going to be $3.
So my loop is going to read through this. It's going to set recycle to true.
shift one thing off, it's going to go back through a second time. It's going to see dash dash.
it's going to shift one thing off and stop processing this part of life
And dollar one is going to be the path to pearl.
So here I've consumed
recycle, I go through the loop again
I see, ah, I got a dash dash. I shift it off and I stop. I'm done.
Now…
if somebody typed an argument to the program, it's in dollar one because I've chewed through that much of dollar at.
So I can say if it's a directory
I know that my build directory is that one.
And I'm going to build there.
Otherwise, if recycle is true
you screwed up because I can't recycle a non-existent directory.
So I puke.
Otherwise, if it exists, it's a turbo.
And I'll validate the trial later.
Otherwise, you gave me a non-existent verbal, which doesn't do me much good.
Otherwise.
You didn't tell me anything at all, so I'm going to assume for the moment
that wherever we're working is a distro directory and keep going.
When I'm done with this, and again, you can see the approach I used here was just defining some global variables
Because there's only one of them. It doesn't really matter much.
So the bill directory
or the tar ball. I can define what are three variables here. I might have set recycle.
I might have set the build directory. I might have set the tar bar.
If I'm alive, now I'm going to go through, I'm going to validate my runtime environment.
So I'm going to validate the curl and make their exist in my in my path.
I'm going to
I'm going to check the build directory.
If there is a build directory.
I'm going to validate that it exists. The reason I do this for loop, by the way.
I say fori in this. You'll see this all over the code.
I use the single letters I, J, K, X, Y, Z, A, B, C.
as throwaway variables.
Technically, they're global, but I always overwrite them because I never use them
past the length of a single
loop. So even in the same subroutine, I'll use I everywhere. It's there to be overwritten.
But the point is then all of my tests are on $y, so I can cut and paste them.
I don't have to put build D here and something PWD here and something else. I just say
For I and build directory, if it exists, if it's executable, peachy, those are the two things I need to validate about it.
Actually, I should validate it's writable there.
And I'm done. Now, if I'm recycling it, I don't need to write to it. So I don't validate that yet.
If the tar ball is not empty.
I call extract tarball, which is going to puke if it can't get the tar ball extracted.
Otherwise.
The current directory has to be writable
or fried.
Then I checked my working directory
I make sure that that's writable.
And then if the build directory looks like Perl.
I checked, does it have a Git subdirectory? If it does, I can call git branch
And make sure that I've got a versioned branch checked out because that's how I'm going to know what version I'm building.
Because that's how I figure out what directory to put this in.
Otherwise.
If the build directory has a version in it, then it came from extracting a tarball.
And the directory name is going to be curl dash
5.40.
something you know it's going to be the tar ball name with that with you know that tar.gz stripped off of
So I'm going to get that.
Then I know…
You might want to…
I know what the Pearl version is.
You might want to add Git to your executable check up above.
You might want to add it to your executable check.
What is? Yeah, I just hacked this today. I'm not done with it yet.
I'm in the middle of making this thing work.
But yes, there's a few other shell things I'll probably put here.
make um
just about anything shell-ish
I'll eventually put up in that loop.
And I'll make sure that they exist.
And the only reason for doing that, you know, the code's going to blow up just as well, because if you look up here.
this thing called execute, it logs what it runs. It checks.
whether the first command exists and is executable, but I'd rather find out early
Before I've wasted half an hour
doing something that I'm going to puke halfway through running the code.
So this just gives me an earlier way to find out that some of the more exotic commands might not work.
I don't think we call that function yet, though, have we?
But yeah, curl maker gif
make or all things I could check. Curl and maker are probably on everything. Get and make might not be, especially make.
A lot of distros don't include make by default these days for the same reason they don't include GCC.
people have forgotten how to build things.
But yeah, that's a good point. I should be checking for more things there. I just
I haven't gotten a chance to put them all in yet.
I kind of thought you'd be using that execute all over the place, but I saw you calling get
without it is the idea you would have all of your balls like that
Yeah. Normally what I would do, well, normally what I do is
Oops.
What I would normally do, well, what you see here
what I do for these
Um.
This is a cute trick. This is an example of
compiling, this is dynamically compiled bash.
Can you see what this is doing?
No.
Curl and make your are the things I'm looking for in the shell, right?
So I'm going to set J equal to which.
So which command returns the absolute path
to an executable that was found on your path.
if which
doesn't return something
It wasn't there.
So I can't run the program.
If it's not readable and it's not executable, it's not going to run.
After that, dollar I is…
CURL.
J is the absolute path to that program.
By evaling it, I'm evaling the statement
Curl equals, you know, being curl, bin user bin curl, whatever the path actually is. This is installing a variable into the code
whose name matches
the executable. So back down here
I say dollar curl this, this, this.
dollar pearl, this is an absolute path.
It also means that it's harder for people to put
malicious code in an off-ball location
And maybe do something to hack my, you know, if they hack my path or whatever.
I'm going to see what the absolute paths are as I'm running them.
And I can say, wait a minute, why did this pearl come from user local bin? Who the hell installed something there?
So partly this security people will do this often that they say you always must use
absolute paths for everything you ever run.
And normally you would say curl equals
and set it to a read only at the top of the code so that all of your shell executables use absolute paths and that way
A classic way of hacking your system.
is someone takes a script that runs as a privileged user
And they will hack its path to include an off fall location.
And then they will put an innocuous looking program in user local bin curl
that installs a back door and then
deletes itself and then runs itself
execs
the real user bin curl with its arguments.
And now the file is gone off your file system
But it installed a back door.
So usually it's a good idea to validate that the absolute paths for what you want are in the right places.
Hey.
This hack does not do that. All it does
is verify
that these things exist in the system
And then I'm not going to puke when I go to run them and then runs them with absolute pads so that if my path
We're screwed up during the execution, I'd still have valid execution.
ads.
I don't do it here, but there are times when you write shell code and you will actually hack the path during the executing of the shellcode.
And then you find that, oh yeah, the path that I need for this cuts off that. And you find it, it just
it's easier to store the absolute paths and know that you'll get the stuff later.
Okay.
And again.
I do a make your here with a dollar maker minus P minus V.
Just so that I know it's an absolute path.
I think you just have pearl on your brain just like putting dollar signs everywhere.
Well, no, I forget, and there are a couple of others. There's get
And there are a few things that run in here that I should have put there, but I hacked this on the fly a couple of hours ago.
And…
I wrote this from top to bottom in one sitting.
I've run into many of the same recommendations about using the absolute path and i
agree and parrot.
I was going to say if you were sanitizing your path.
Well, what you would do is this.
Hold on. Hold on.
Here.
Can I finish?
Oh, sorry.
But you spoke to the fact that you spoke
you're using the witch
to finding the path in finding the path
it does not protect absolute. Somebody could still mess with it before you run.
But it does protect
running breaks your path.
Okay.
a few things maybe.
So you answered the concern
Well, here's what you do.
you do this, he upgrades
Up here at VIN or user bin.
And let's just say
Um.
I'm only going to run things that live under bin and user bin.
Period.
Thank you.
Okay.
Thank you.
There you go.
So basically, if I don't run something that begins in bin or user bin.
it won't run. I will be denied the option to run it.
Okay.
So what this says is basically if everywhere I use make
Let's see, make.
There we go.
Thank you.
Okay. At this point, all the shell stuff that I'm running
is…
is only going to be run out of bin and user bin.
or let's say opt-in because I know I have stuff in there
Can you see what this is doing?
So here.
Hey, how come you're able to put that ore on another line?
Does that work?
Oh, no, that's called a typo.
Oh, okay. I was like, how are you doing that?
Yeah, no, you don't.
But basically what this says is if the thing I found
Well, actually, I would put this up here so
start getting a witch
If it doesn't match my path rx
Duke.
Okay, so now my curl maker make it. These are all the sorts of things. Now, I know
for this
This piece of shell code only runs as myself.
Unlike a lot of people, I'm not stupid. I didn't lock down the ops directory so that only Root can access it. So I don't need to be super user to maintain most of this machine.
If you're a bin user, you can install things into opt and user local.
And what's important about that is there aren't very many people in the bin group
But none of us has to do an SU do. So you can always tell who did everything.
Because the actual correct username
is associated with everything that gets installed.
The other important thing about that is because I'm not super user.
If I run something and someone tries to run malicious code to install things in bin or user bin.
They can't. It's going to puke.
So because of the way I set these systems up.
The…
with graduated privileges.
there are a group of people who can install things into opt
They can't install things into Bin and SBIN and user bin.
or anywhere under user.
basically they can't write to the stuff that's important on the root volume.
But that makes it a lot easier to run this kind of stuff because this is extraneous.
if somebody did something malicious, probably all they'd be able to do is cause my code to puke by
by not being able to write to the wrong place and I'd see that it was there.
A lot of people lock things way too far down.
And as a result, you see all the
the examples in the books to have SU do something something in
If you're SU doing everything everywhere, other than starting something on a port less than 1024,
Or, um.
or a thousand, excuse me, or trying to start and stop demons that have higher privileges.
And most of the daemons give up the privileges. You're just as well off
having a user postgres
And giving people the right to SU do to the Postgres user and use the Postgres user to start and stop everything.
done. If I have shell code that is suid postgres.
The only thing it can do is start and stop the database. Somebody could include malicious code to delete everything.
But they can do a lot less damage to me than if they're SUID to root
A lot of the idiots that run the
have written generations of code assume that the user ID is root. There is nothing special about the name root. I know we're going to talk about this once before.
Zero is the magic number. If your user ID is zero.
you have super user privileges.
So the classic way somebody hacks your system is they put something in the middle of the password file
It's Joe Blow, and they just changed Joe Blow's user ID to zero.
And there's still Joe Blow. If you do an LS
Everything on the file system is owned by Joe Blow.
But Joe Blow is a super user.
There's nothing special about the name root.
Lots of stuff now checks for the name root.
And so you have to be root
And root is, you know, if you leave, for example, an SSH server
If you run the daemon running on port 22, you will spend a third of your CPU time doing nothing more than getting brute force attempts on the user ID route.
Simple fix. Well, other than changing the port
take the name root and call it bugger off schmuck.
cram it into one word and stick it at the top of your password file.
Nobody's going to try and fake bugger off schmuck. They don't know it exists.
It is a certain level of security through obscurity.
But it certainly is better than having root exposed because everyone knows it's there.
But anyway, yeah, this is a quick and dirty way to make sure that everything's just run the way it's supposed to be.
What do you think about instead of failing, because those aren't installed, what do you think about installing them?
No, no, no frigging way.
Because I've seen some scripts that just install all their dependencies
Three reasons. There are a couple of reasons. One is, I just told you, I'm running this as myself.
I don't have privileges to install stuff like that.
I want to separate the privileges.
I am in a group.
that has the ability to write to the opt Perl directory.
And let's say opt postgres.
Maybe there is no other directory that system directory that contains executables that I'm allowed to install into.
That would be smart. Now I can have my dba
be a member of the postgres group
And that person then has access to the Postgres files without us doing
Yes, you do. They have access to
Opt Postgres where the stuff is installed if they need to maintain it
I might go into the SU doers and make it so they can SU do Postgres
Without having to give a password.
But they don't have to be super users.
You want to dick around with the Git, you're going to have to do an SU.
And this doesn't. And I don't want to escalate the privileges of something like this
It's a one-off hack.
Because there are too many ways, like somebody noticed I hadn't checked get
this, you know, code like this that's a top to bottom install script
is taking the time to make it boilerplate enough
that it was safe to run it as super user would take me forever.
I need to get this done.
So the first step to doing that is making sure that it can't do anything unfortunate if I screw it up. It will puke. It won't wipe out the root volume.
is this might do an RM minus RF to clean up a build directory.
And the only way to do that is separate the privileges so that it doesn't need to be run as super user.
Did that make sense the way I said it?
Yep, sure did. Can we see where you're calling the execute method? I don't think you've seen that yet.
Oh, execute.
Right here. So for example.
Well, there are better examples.
So here's a good example.
And this gets back into a race. This is an array.
This installs the CPAN M executable.
CPAN M is a package installer.
The nasty thing about package installers is if you have to install them, you need a package installer.
to install the package installer that you have to install the installer to install the package installer installing the way
The guy that wrote this got out of it. The actual program
Oops.
Thank you.
If you look at this.
This is a cop, this thing is the version that
Basically, the program arrives
fat packed.
And so what you curl
is something that can run
it can execute itself and it executes itself to extract itself. So it's a self-extracting package installer.
So if I grab it off the net and I pipe it into Pearl.
it will execute and install itself.
How's that for cute?
variant of a show.
So to install it, what I do is I run dollar curl
And I give it the argument and I give it a pipe.
And then I run Perl.
And Pearl is running its standard in as the program
The double dash, because Perl uses get opt
says pearl
Do not treat the remainder of this as arguments to the curl executable.
Leave those on the stack.
for the running code that Perl is executing.
Do you see what I'm saying there?
So when Pearl runs get opt
pearls get up is going to see dash
And a bear dash pearl says, ah, I know what that means. That means I'm going to read my executing program from standard in.
Oh, here's Double Dash.
I'm going to stop chewing through the stack.
I'm going to leave what's left
inferals argv
I'm going to leave the rest of the stack alone
for the code that I'm running.
And then this thing
sees its own name and it says oh
You want me to install the CPAN minus application.
And then it goes out to CPAN and actually gets a copy of itself and installs it properly.
So this is an array.
Purely because if I try and put this on one line, it wraps.
So being able to break it out this way, I think makes these more readable.
And you'll see…
where I do the configuration
um
config args
up here. This is another case. These are all the arguments I'm giving to the configuration execution.
But I find it easier to read these with a little more white space, line things up.
with the ones without arguments at the front do whatever
And then I run this
I find config
And right now I'm at the point of testing this.
I'm done testing that.
And then I run the configure that I found with its
And I could pass this to execute.
So execute is going to log that.
And then if it puked, it would tell me.
So execute is going to log that. And so when it gets to the config args
Is it going to put like two tabs of white space
Before every…
No. No, it doesn't need to.
It just runs whatever it was given.
Well, what about the well because it was like in an array
What's that?
This thing is going to print out dollars flat, which is all of its arguments.
Yeah, what's…
it's going to check that the first argument is executable
that exists that the first argument is executable
And if the first argument has non-zero size.
Any one of those things is very likely to cause this to fail.
But at least this way i get a
a more reasonable error message from it.
At that point, I eval running whatever that was.
That's going to take the bash compiler
it's going to expand that text
it's going to parse it in the normal way that it would, the same way it's going to pass it through the same parser and compiler
that this line went through. And whatever the result of that is, it's going to execute it.
And if it returns, now I use this for executing shell commands
So if it returns non-zero, I say it was fatal
And I say, what executable it was
And what the error message.
But in that array, though you had
This is good enough for something running single stream.
Yeah, so in that array, though, you had a bunch of white space before each item.
Right.
And I was just wondering if that will be printed out in the log or if that's like trimmed.
No, that gets flattened out.
what happens, okay, what happens is
Um.
Thank you.
what happens is the parser, again, bash
is bash is an interpreted language.
Every time Bash sees a line
it has to figure out what it means because
you've got something like this
Where I've got it's executed this config dah, dah, dah, dah
I could just put this.
This would execute just fine.
Because the compiler, the interpreter.
would hit that line and say, oh.
I see a dollar sign, I will expand that.
I see a dollar sign. I see a curly brace. I will look inside the curly brace. I will find a symbol.
that's got a star in it. That says, take everything in this array and paste it together
with a space.
And generate one long string out of it.
I have two.
Now, when the interpreter read this, it discarded all the extra space
So the first argument is minus DES.
The second argument is this. The third argument is that. The fourth argument is minus the pager. The fifth argument's is whatever is inside these quotes.
And then…
This one is the Pearl directory, and then this one is whatever's in these quotes is one argument.
all the other white space was discarded when the interpreter
tried to read in the definitions
of these lines.
There's a variable called IFS.
input field separator.
Which is a string.
Um.
Okay.
And you'll see that it's a space
and a tab and a new line
um
Well, yeah, the Echo put its own stuff in.
But 2027 zeroa
our space and tab
and new line.
And this here was a vertical tablet. 09 is a vertical tab.
Oh, God.
machines that don't include all the man pages
09, he has horizontal tab.
New line is 0a.
all type.
Space is three, two hex
No, it's 32 decimal, excuse me.
useful.
Space is 32 decimal.
And I think in vertical tab.
is zero bravo.
Form feed is zero, Charlie. I think that's actually zero Charlie is what's in there.
But basically what happens is whatever the contents of that variable are.
As Bash is reading through the source code.
it will break apart the line by discarding the contents of this and using them as separators.
And repeats are just separators.
If that made any sense.
So…
Tab.
new line space tab and new line.
two zero is heck is
you can't see the space there but
tab space and new line.
are the normal separators when
the interpreter is executing.
There are times when that is handy to change.
There was one time I was writing bash code
And it had to read a massive
piece of json
as a single item.
into an array.
And what I did is I separated the elements of the array with tilde's.
and set IFS equal to tilde.
By doing that.
When there was a subroutine that had a line in it
uh…
And it had a local IFS equal this.
And then it had…
you know, big structs
And then it was just a long quoted, you know, it was a long string
of stuff that had something and a tilde and something
And it killed a…
And what I got was all this stuff in between unmodified.
And it allowed me to have basically it was a, actually it was a here script
I hear string. So it was big equal
Thank you.
Okay.
Something like that. And then if you see now here scripts work
If not, there was a thing I did before on it. But there was a bunch of lines with the word JSON on the left margin below it.
And that huge thing was just one piece of text that I could cut and paste out of whatever crazy place I got it.
But…
The only thing that broke the elements apart were tildas. All the new lines and tabs and white space inside of it were ignored
And each item had all of its white space intact because the separators were tilde's.
you almost never have to change this.
Um.
But what happens here is
Because IFS has got new lines and spaces and tabs in it.
all the spaces and all the new lines are discarded
and use the separators. So whatever isn't white space
between these parens
Each of those is an individual argument.
Did that make sense the way I said it?
Yep, very thorough. Thank you.
So now what happens is I get down here
And I execute
Um.
Config and this
Yeah, it can be one long string.
It's going to check that the configuration is a non-zero sized
readable executable.
If I get an error, it's going to print it as being from this.
But then these are going to be passed to it as one long string
And then evaluate it and pass down to the show.
Now, what I might have to actually do is this.
Because this is going to be evaluated twice.
These double quotes are going to expand once
When this line is executed.
This wine is going to expand to
The path to that config executable
dot slash configure.
Followed by all the rest of this in single quotes.
When that means that it won't be broken up and reinterpreted by the shell when it gets to that eval statement.
that eval statement is then going to get all of this as one big thing
And it's going to push it all in.
And that way it'll preserve all the quotes that are in there so
This will remain as one object.
on the stack dollar
this is this is going to be dollar one
the config dollar two dollar three dollar four dollar five dollar six
all of this.
If that made any sense.
So when you use eval because you're double interpreting things, you have to be
really careful about how you quote things.
That's why, and again, there was a talk I did, if you look it up on slide share
on dealing with that and I can do it again if Gary needs somebody in another month but
The trick here is
You can use set minus vx.
when you're running the code and it will show you every stage of parsing this
And when you're doing stuff like that with evals, it's a good idea to run it
And you saw that before like this thing
What I'll do to check it is I'll do a set minus dx.
Here.
And then…
exit one.
And exit one is non-zero, but most of the time, if you do an exit one.
it's supposed to indicate
something more innocuous than like 255 exit minus one is 255.
So an exit one, like if you exit from a help function
If you exit with one, somebody knows you didn't puke, which is minus one, or larger than one.
But you also didn't finish.
So it's like the difference in a warning and an error.
So am I remembering right that single quotes are literal?
Single quotes are non-interpolating. Double quotes are interpolating.
So double quote dollar purl d double quote
is going to take the Perl directory
and expand it from the variable.
And leave it as one string.
Actually, these quotes here
Those were extraneous because i know
that any director, anything in the file system
isn't going to be broken up into multiple pieces or it wouldn't be a valid file.
But what about the quotes on
The oldest trick in the world is you drive some Unix newbie crazy by creating a file called readme space.
Oh, yeah.
Hey, remove it. No, remove it. And eventually they learn tab completion. They just type
you know rm r tab and it just gets rid of it, they don't see the space
But people that try and type it or cut and paste it, it just never removes because the shell parses out the space. They go crazy.
or create a file called remove space me. And until you learn quoting, it fries you.
But this is not interpolating.
So if I had a variable in here, let's say I had, you know, dollar foo
the actual literal dollar foo would be passed out of that.
Whereas here this is whereas
Okay, so what about line 277, though?
Where we have the double quotes on the outside and the single quotes on the inside. So does this mean
They're passing the passing
That is going to take, well, this is interpolating, right?
This is going to interpolate all the variables in it. If I had, let's just say I had
Oops.
this, right?
Double quote says, preserve white space, preserve special characters.
From here.
um
From here to there is one argument. This is going to be one
string stored within the bash structure. Does that make sense?
Yep.
It's not going to be broken up into separate elements. It's one string.
Right. Dollar one.
The contents of that string
The first thing Bash is going to do is look at in its symbol table and find that config is defined.
If it's not defined, you're going to get empty.
But it's going to find config and it's going to expand the value of config
it's going to find a literal space
it's going to stuff a space after whatever it expands it to. It's going to find a literal quote. It's going to stick that in.
it's going to see a dollar sign, which means expansion time, but now it's going to see a curly brace.
So it knows there's a little bit more going on here.
So it's going to read, it's going to say, oh.
I find word characters.
So dollar open curly word means this is a variable
The square bracket tells it
I'm looking for something that is
either an array or a hash.
we see a ray or uh
He calls them associative arrays.
So whatever it is.
The star says take its contents. If it's a numeric array, you're just going to get the pieces.
you know zero one two three four five so it's going to get just stacked together.
if this was a
An associative array, you get the values in the lexical order of their keys.
And you get back this list.
But you skipped over the single quote, though, right?
So each element of config args is going to be expanded one by one by one by one by one by one by one by one.
And then I'm going to stick a, there's a literal here, so I'm going to stick a literal quote on the end of it. And that entire string is going to be passed to execute as one argument.
So it's going to have one argument dollar one
Now, I don't want that.
But hold on a second though.
Oops.
So how you had it before, right? It has to double quote. It gets to the
Right. This is dollar one
It gets to the single quote.
This is dollar one.
Right.
And then how come when it sees a single quote, it doesn't say, oh, there's a single quote
Because it's inside of the double quotes.
This is one character string.
the rest okay so if you have a single inside of a double
Double quote starts an interpolated string.
Even if there's floats inside of it.
This is one interpolated strength.
Gotcha. And so when
At that point, the quotes are literals
that are interpolated into the final string as literals.
the dollar sign inside of the interpolated says.
Figure out what
The dollar sign
Well, an unescaped dollar sign inside of single quotes
says whatever follows me is a variable.
So for us so good?
No.
I'll see.
Now, it might be a variable like this
Okay.
It's just a simple variable. It's plural d and you look it up in the symbol table and you get a scalar back and you stick it into the string. It might be something a little more complicated like this thing.
Okay, but what about this? Because this config args is global, right? It's a global array.
So if you took…
Right. No, it's not. It is not. It is not. It's not global.
What do you see in front of it?
typeset, but you're not passing any r stands for read only and A is array, right? You're not saying anything
Make a local.
Right, right. If you typeset something
inside of a subroutine, then it is local to that subroutine.
It's automatically local. Okay, but if you made that global
Period. Yeah. If you define
If you just declare a variable without typesetting it or declaring it local.
And there's an alias for typeset called declare that is, I forgot what language used that.
if you
formally declare a variable.
Using local or typeset inside of a subroutine.
then that variable
is local. You can, if you look in the bash man page
Um.
Okay.
There we go. Typeset minus p nic equal value now
Each name.
This is an indexed array, associative arrays
I can typeset something to a function and say, is somebody Christ created a variable with this name, puke
You can create things that are integers.
And it will only take things that look integerish
You can have a variable that always stores things as
Now, you can say something is lowercase and read-only, so you can assign it once.
it will be lowercase and done.
Tracing is for debugging. Every time you touch it.
it'll call a trap that will dump out the variable.
This will uppercase everything.
And…
Minus x is minus x.
export, which says if you fork
Anything that's if you do a typeset minus X or you type export and a variable name.
equals something, then that variable will be pushed into the environment of a fork.
And…
The other thing you can do, they have indirect variables too is another thing you can declare.
But you can declare a variable that every place it's used, it references another variable.
Which is great for dynamic code.
Um.
But typeset
is local. So for example, I couldn't do a typeset minus, if you look here in
Um.
here in the command line. I could not say here
It's…
If I said kiteset minus r recycle equal one.
I would fry myself because now recycle wouldn't be available outside of this subroutine.
Did that make sense?
Yeah, you got to add the dash G.
Um.
Yeah.
Okay.
Thank you.
Oh, there we go.
Ah.
Yeah, see, it doesn't list it in the list of arguments, but if you look up in the
like two sentences up dash a
Yeah, you have to look it up elsewhere you
Yeah, you find, yeah, you have minus g is global
And one of these is used
for references.
Stop. Okay. The G is mentioned right there.
Thank you.
Okay, but so…
Okay.
AFR, no. Well.
Stop, stop, stop.
AF I L R T UX. It would have been nice
It's not listed but if you
More along.
If Stolman had put the rest of them in here
But…
Even.
Yes.
Scroll down about four lines.
I get it.
Stop. Okay, right there, right?
Unless the GA option is supplied, if a variable name is followed by the equals value is set to the value. I get it.
Okay.
But that doesn't say what global does.
It's actually two sentences above the top option. It says what it does.
But it doesn't matter.
Yeah, but it's not down here in the middle, which is what makes reading this man page.
such a pain in the butt.
Yeah.
Okay, you're right.
Yes, you could do a minus G on it.
So…
And that was how you could do it there.
at a certain point, trying to do this in hopefully less than an hour, which I've probably way overrun by now.
Yeah, by 100%.
So the dash, but if it was global though
then you wouldn't have to quote it at all, right? Because you could just pass…
You can only quote it in…
Double quotes.
I know, then you could pass. No, okay, let me rephrase. So if that config args array was global
You could put it in single quotes without the double quotes on the outside.
And they would pass the literal string
dollar sign, config args you know
et cetera, et cetera, into the execute function
And since it can access that variable, it would expand it when it's calling
during eval.
Uh-oh.
All right.
Um.
Yes, you could do a typeset minus E and do all that minus R, and that would work.
But when you start doing that, people go
Steve, we lose you?
I know it keeps muting me. It's trying to tell me something, I guess.
Um.
Your audio is getting really weird. It keeps cutting out.
Yeah, I'm missing…
It keeps muting me. Well, somebody who's the fucking moderator just click the unmute button for me?
It doesn't like me keeping myself unmuted. It's getting confused with life. I don't know why.
It's not showing you as muted.
But it keeps telling me that it's you are muted anyway.
Um.
than it most hard work.
Blame it on gremlins. Somebody didn't do the donut dance.
But anyway, does the notion that this is
This…
is going to end up as one scalar string
on the argument list to execute.
Thank you.
The double quotes mean that everything in it
will be interpolated as part of the result.
Anything that starts with an unescaped dollar sign
is going to be treated as a variable and the contents of that variable will be expanded, interpolated into the string.
Anything that isn't a variable is going to be passed through as is as a
as a single quote. Now, one thing about double quote, a double quotes
I could put a tab here.
And that backslash t is another thing that gets interpolated along with the variables.
So I can put a literal tab in there if I thought it looks pretty.
In processing the eval.
that tab would be stripped out because tabs are part of IFS.
And the eval would say, tab, ignore that, use it as a separator. So it wouldn't matter.
But this execute is going to get two arguments.
config and whatever the glommed up result of these double quotes is.
The trick is it's going to arrive as a single quote.
That is still one argument. It just doesn't have the single quotes in it.
The point of the single quotes
is here.
If you look at it.
Dollar one is that configuration executable, right?
Thank you.
When it does the dollar star here.
I'm not actually using dollar two explicitly.
If I'm not using…
then actually probably I can get away without them.
That's the kind of thing you've got to check with these evals.
Because there are
single quoted strings that are nested inside there.
So I may want, yeah, eval…
I wanted to treat all of dollars flat as a
as a single argument
I might get away with it. I think I actually might be able to get away without the extra quotes. Evals get messy that way.
This, for example, dollar jobs make minus WK this, this.
This is not passing it through execute.
Thank you.
Because I wrote this line of code before I wrote the execute function.
And because it kind of breaks the way execute works, because dollar one is test jobs equal jobs.
Which is not what I want to report the error as.
So here I just ran it.
And then…
Pearl D, bin pearl.
Um.
This just checks that the pearl was installed.
So this says, okay, the thing that I thought I was installing got installed
The VIN directory is the name of that, and I execute h to ph
And again, this here is an absolute path
So if somebody has H2PH in my user local or something.
it's not going to affect my life.
And then I'm installing the utilities, which gets CPNM. And notice here there's a pipe
So that pipe is going to be passed through as a literal
So this is going to expand to curl minus L this pipe curl, blah, blah, blah.
So this is actually going to execute curl piping it into Perl.
Does that make sense?
Sounds confusing.
No, if I had typed here
curl minus L.
Okay.
Okay, everyone knows what a pipe does, right?
Yeah.
Well, this is going to take curl
and put the output of it into Perl.
This is going to do exactly the same thing when it gets to the eval. Eval is going to take these as four separate arguments.
Lomb them together with a piece of white space each.
And then execute it.
But this time the pipe
is going to be part of one long string. I had to single quote it there. Double quotes will work too, or I had to escape it.
So that the shell doesn't try to read the pipe
as a valid meta character
And try and complain that as a pipe in the middle of a variable definition.
So inside of the quotes, despite there's just one more character.
So then I execute this.
And then I checked to make sure that it exists at that point.
I can run CPAM to get CPAM outdated.
And then I execute
CPAN outdated.
And pipe the result into CPAN M.
And that takes all the names of all the modules that are more recent in CPAN and pushes them into CPAN M where they get updated and reinstalled.
Which is how you keep your
your stuff up to date. At least we'll see that am.
And then I have to get to the modules, which has to take my existing Pearl
Which is the one on my path.
And then…
my modules, there's a module called xdutils installed
that will…
get a list of modules that have been installed since the release was created.
And so what I do is I execute
a thing in Pearl to avoid what Larry Wall calls leaning twigs syndrome. You've all seen this. If you want to interpolate, if you've got this.
and you want to have, you said…
Thank you.
Okay.
Thank you.
You get that kind of mess.
Because you have to escape all the double quotes inside the double quotes to get them to be literal double quotes.
To avoid that, Pearl has a notation
QQ…
is a double quoted string and you can use slashes
You can use pipes. You use anything that doesn't appear inside the quotes.
And then everything inside of it
until the next one of these
that's my strength.
And then you don't have leaning twig syndrome with all the backslashes because then if you have something double interpolated, you have to escape the escapes. Then you have to backsplash. It's a mess.
So what this basically allows me to do is have a double quoted string inside of it without having to put a quote that's going to be interpolated funny by the shell.
So this is going to list, this is going to create an object
and call its modules method.
And if I take that
What this does is it executes this
Oops, and I just noticed a bug in the code.
Okay.
Drop you again. Oh, fucking peep.
No, what happens is the
This is a list.
So actually, this…
is a subshell this says
And you can get the same effect in Perl if you type
QX…
blah.
You can see this in Perl.
And QX is the same thing as they're like back ticks, but it saves you again from from quoting
problems. But this says execute this using the shell, take its output
And hand it back as a scaler.
And here I just store it in a variable. The same thing happens in bash
If you have a dollar open close per end.
that says run this
and return its output.
In this case, I'm storing it as
So I can say actually it should be typeset minus RMOT r
makes it clear what I'm doing.
At that point, this gets converted to a list of single words.
At that point, dollar mods without any array stuff on it looks at the first element.
Well, if there's nothing in the first element
something went wrong.
teachy at this point, I can take each of the mods expanded as a separate word
And I install each of them one by one.
And I could do execute on that too, but CPANM has a lot of its own logging and it will tell me enough about what it's doing that I don't need to go that far.
And this will take all of the modules that appear in my current path version of Perl
And it will reinstall them into the new one because this cPan M, it is an absolute pass to the Pearl I just installed.
If that made any sense.
the pearl that is on my path
which the way the paths here is set up
you'll see this opt bin at the front of it
That's a bit of a hack I created 20 years, 30 years ago. Not so long ago.
So…
You'll notice that everything in this directory
is a SIM link.
just about. This isn't.
But what happens is
This takes the part of Postgres and Perl and whatever else on pg bouncer.
This puts all the things we actually use
in one place. So instead of having to have opt postgres, bin, opt, purl bin, opt GCC bin, opt g++ bin, you know.
you just take the portions of what you need and put them in Hopkin and SIM link them into dot dot something bin.
And it makes it a lot easier to maintain your path because all it has is opt-in at the front of it.
So this is going to go through and install all the modules and then I'm done.
And I'll usually put good night at the end of the
I've never seen any other programs that use good night
So if I log that, I can tell that it's my program ending in a log with 50 programs dumping into
the system logs.
But anyway, I can do
Sandbox build pro five
That is a Git repo.
Thank you.
Right.
And CentOS is so ancient can't do no less on a directory, but
At that point, I can do dot slash init db
Sandbox build pro five. Let's see if this works.
Oops.
Oh, what am I doing an EVB?
Okay.
Thank you.
So this is now running the configure.
Thank you.
Okay.
There's a funny cursor. There it is.
So this you'll see here, install Pearl, building in this, my working directory, install from this.
Pro version is 540, which I know is correct. So it's going to put it there
Installing the pro 4
Remember, I put that set minus vx in there so I can see what it was doing.
And that did what that showed me what I wanted. See this eval here?
eval configure and you see how all these get expanded out
Now, what I notice is I don't have the quotes that I wanted here.
Probably what I need to do
Um…
Thank you.
Um.
Yeah, this should be that.
And this should be that. There's nothing to interpolate, so there's no harm in putting double quotes. Otherwise, what I'd have to do
is have it this and then I'd have to escape these
Okay.
Sorry about all the typos. They rebuilt my spine out of spare parts from dead people a couple of months ago, and I'm still having problems with coordination.
We did some.
But because these are all literals.
there's no harm
in putting a string of literals in double quotes.
So this is going to preserve the single quote. So when it's expanded inside the eval, the eval is going to get it as a single quote
entry.
Thank you.
So you'll be able to see that here. We're sort of exiting. Let's just reset.
Minus.
Um.
Did I hear that right?
Yeah.
Yeah, it goes pretty fast.
Thank you.
Um…
No, this is just doing the cleanups now.
So now…
Yeah, you'll see here, you see where it's got this
escaped quote here.
That's showing me that the quotes were passed through to the eval.
So if you look here, see what says this thing?
Do you see now here, when you look at this.
you see that you can see the single quotes are around this
And the single quotes are around that.
So these, when you look at it in the expanded version.
you get this
escaped garbage, but this is showing me that those quotes
made it into the eval statement.
So you see here, this is the actual line that's going to be executed.
configure and you'll see now
Um.
Page here is user bin less minus RM no init, but now this is single quoted
So it's one argument to the minus d pager.
And the prefix has no quotes.
and optimize is minus O3M mark equal native pipe and
don't my case about minus O2 versus minus O3.
But then that's going to be passed down as a single literal.
And this is all of the sanity checks that Pearl does on the
Okay.
So you can see now it's compiling everything.
On my server here, it takes about
three and a half minutes to compile term.
Earl in another five minutes to test it. One interesting thing about Pearl, by the way
that is different than Pearl and Riku.
that are different from pretty much every other language
If you install a module
with curl the tests are actually run in every environment.
And that uncovers a lot of bugs because you can't
Nobody's got all the machinery that at least Pearl runs on.
Including VMS, I just don't have access to a VMS server.
So there's a neat thing called cPanel reporter
that sends back the results of that to a central place
where the errors are grouped together. So every night I get a report
of all the places where my stuff failed to compile on some weird ass operating system or some
more modern set of libraries that I might have
And that can be a really big help in keeping track of things.
But when you run Perl to test it, there are
13,000 tests that it runs before it installs. I mean, it's a
hideously large number.
when you get in all the tests and the subtests and all the rest of it.
This is running on a two CPU VM in
in AWS.
If I ran the same thing here
you can see one thing.
This is actually a cute trick for, you got a couple lines.
Um.
Here you go. This may be hard to read but
If you go into the prox CPU info file.
and you grip MHz out of it.
you're going to get one line for every processor. See, we're into the tests now.
If you count that.
That is the number of cores that you have available.
Which is a really good. So what I then do is when i run the make
You'll see this is jobs.
I'll make minus jobs
which runs that many in parallel. So basically I'll swallow all the cores.
And I do the same thing here with the tests.
the
the number of test jobs running is jobs.
I could do this by exporting test jobs and then running the make as a separate option.
But that's why I didn't use execute because dollar one would be test jobs equal jobs, which is not
not very helpful.
And if you make all tests install in that order, you could do make install and run the rest of them, but
then it's a little more explicit if it pukes which layer it puked in.
Thank you.
And this is running about a third of the way through the tests already.
But this is what it looks like building Pearl.
Thank you.
Did somebody say something? It broke up.
hopefully I sound better than that to you guys.
Yeah, no, I didn't hear anything breaking up, so I don't think
Okay.
Oh, I'm hearing some background noise of some sort then.
but um
But yeah, you could
Beware, by the way, if you write out a running piece of bash code.
bash will reload the file and reinterpret it from the point of the offset it was at.
So you can be working on a bash script and accidentally save it from the editor.
and break a running piece of code.
But we're right up to here now where it's
It's building…
So it's, yeah, it's right here.
And then the next thing it's going to do is sanity check
whether the Perl directory has been Perl is there for that file
Does it exist? Is it executed? Does it have non-zero size?
If so, then my purl is equal to that thing.
And that's the
But that one isn't on my path yet because it's not sim linked into opt-in yet.
Eventually, what I'll do
is…
So you'll see right now there's 3.41 is where everything lives.
So in a little while, what I'll end up doing is RM starv
5.40.0 slash star dot.
And that is how I upgrade the version of Perl.
If I want to downgrade it.
I link 534.1 start to dot.
And then what happens
Because…
opt-in pearl, well.
That actually should refer to opt
pearl bin
show up.
That got screwed up when it got put in. That was another script I did that didn't do it right.
Oops.
Those should be SIM linked. It's just purl, bin, purl.
And that way, if I upgrade it here, it upgrades it on everyone's path in one step.
But…
You should adjust that, actually.
Excellent.
Thank you.
Okay.
Thank you.
Yeah, see, this is done the right way.
What the hell was I thinking when I did those?
Thank you.
32 in Beijing.
This is a neat notation, by the way.
This is part of dealing with the shell and bash.
This is a list of alternations.
This is going to be expanded to all of these.
And these actually can have multiple alternations within them
And we've all dealt with it. You've got something where you've got 800 subdirectories and there are different levels and everything else.
With these things you can
Okay.
Thank you.
Okay.
So you can see where this is going to expand all of those. If there was pearl
This could be pearl, bin, live.
And it's going to expand all of those.
If you had Pearl being live.
Um.
Okay.
So you can see how this makes it really easy
to generate pretty large lists of things that have subdirectories or subtypes or files that are in multiple places.
with
pretty straightforward notation because anything inside the curlies is just expanded as is.
You can also do
So it'll do alphabetic and numeric increment.
with back dots, which is again is a nice way if you wanted to make
you know classic thing is you break a directory down
And you keep the files in a reasonable space you take
the first two letters of every file
And then you put every file in a subdirectory under its first two letters.
to a file ABC would be in the directory A, subdirectory B is ABC.
So I can do um
That'll make it interesting.
Thank you.
Okay.
Okay, that's how hard it is to make all of the subdirectories.
Thank you.
Bye.
um you know it's
things like this
Because again, that could have been
user being lied something. You can nest subdirectories, you can make fairly complicated collections of directories
Can I push it down?
um
Okay.
This is another nice thing.
Okay.
Thank you.
Hmm.
Oh.
Thank you.
Yeah, there it is.
One of the classic things you want to do is keep just enough information going on the screen.
So that you don't get a timeout for not having any output.
But you don't want the screen to scroll forever.
So what snooze does is it prints the current time
And then every second it prints the time until it gets a return and stops.
What's sexy about that
The cue trick here
is by doing a carriage return
And then the date and a carriage return.
What that does is it prints the date at the left justified on every line.
If you've got something running in the background and it generates output.
This carriage return will put you at the start of the next line
And then print it. So you won't get the date stuck on the end of a line. They're always left justified.
And then anything that comes out is going to overwrite the current date and go and then the next date's going to go below that.
So if you get stuff appearing on the screen, you can tell when the hell it happened.
Because every second, this thing's going to update the screen
But with a one second timeout.
If you hit return, if you type anything
you know do a read and it'll break. It just ignores the input
But until it sees some keystrokes.
it's going to sit here. I can type x
And you can see if there's some output, it'll just keep going.
If I hit return, it exits.
But I do this. The first one shows me when I started. If I just walk up here, I can tell how long it's been snoozed.
And this will keep the terminal up
You know, in Amazon indefinitely because there's outputs and they're not going to close it
Ooh, we're getting through most of the tests now.
But anyway, I don't know if anyone wants to watch the rest of these test runs because it gets kind of boring.
but there are
thousands and thousands of tests that Pearl runs
every time
it doesn't install. And then all the modules
run their own tests every time they're installed.
So if people write decent tests for the modules, no matter how crazy your particular environment is or how customized it is.
you're sure that the stuff has at least been tested in your environment before you try to run it. If you look at every other language, Rust, Python.
Scala.
If you install modules.
they all just plop themselves on the disc and they're done. They don't actually run all of their author tests.
And you can see here some of these are skipped because we're not on Bios or we're not on VMS.
or we're not on Windows 32.
So unnecessary tests are skipped.
Some of these are skipped because they're only run by the authors.
then they have to have a special switch or they just don't get executed.
But they make it
a lot safer to run this because everything is examined before you use it.
Anyway.
Gary, I promise I'm going to go back through and clean up all the typos and oops.
In this thing.
And…
But you did a wonderful job.
putting stuff together i mean it it uh
It was a great attempt.
Well, thank you.
technical stuff.
But yeah, this was…
I wanted to get some more stuff in here. I just don't know how to write it.
We're about four hours on this last night.
There's…
where are all the bugs?
Was this helpful to anybody?
Definitely.
Yep.
Well, for you, Gary, was this helpful for how many people are awake? Just everyone unmute yourselves and at least say hello. Come on.
Arrow.
You're going to lake.
Hello.
Oh, we got a couple of people awake at least.
Um.
Yeah, I got to go back and proof this.
Okay.
Um.
typeset minus r locals minus rv where were all the
Here we go.
Let me suggest, although the the
we give up. Nobody wants to watch me proofread this, I don't think but
Unless somebody can help me find the errors.
I think a few people were…
A couple of these had dollar ones in them.
We got a couple more left.
Lord equals.
your Eastern time zone and for them, as for you, it's getting to be 10 30 so
Thank you.
We probably should.
Good.
wrapping up here. I would like to
Thank you.
Okay.
make a couple or a couple of
brief announcements. First off, very apropos
to your talk tonight. On last Tuesday, the 12th of the month
There's an entry on our calendar and also we've got a little discussion between Denise Dean and myself and discuss. Professor Tom Kurtz
passed away. He was a professor at dartmouth
And he was one of the co-creators of the basic language
He was 96 years old, I think, when he passed away.
Hmm.
And the other
Wow.
that and
isn't hard to have him speak?
Yeah, yeah. But the other reason why it's very apropos to your talk tonight is the other thing that he was co-creator of was the Dartmouth timesharing system.
That, of course.
Oh, wow.
done on a GE mainframe.
And the GE timeshare developers said, oh shit, the academic people seem to have a
a lot of things that we don't have in the commercials
attempt at timesharing. And so it was really the integration of the Dartmouth timesharing system
with the General Electric commercial product that became
Timesharing on G-Coasts, which of course is what was copied, emulated
by the original Unix developers. A lot of the stuff that was already
Well, actually, it was first
The first place it appeared was LPARS.
So the first people that actually made use of it was IBM mainframes that's
Time sharing on those systems became the little boxes that LPARs were built out of. If anyone here has ever played out of
thing with IBM.
And then the funny thing is the time sharing on Unix took a completely different tack
Mm-hmm.
that was a lot less protective.
And then people realized that was bad.
And we've since then we've had
Charoots and jails and everything and every, you know, all the different you know
son had a version and Charut and
Finally, what's interesting about linux
you know coming
70s, 80s, 90s, 50 years later
What's fun about it
is that the
this the
current system on Linux that allows for Docker and all the rest of these.
is actually implemented from IBM's LPAR system.
using process groups and so instead of isolating the process, they isolate the process group.
And that whole design was taken from IBM's LPARs that were taken from that Dartmouth layout.
So…
I'm sorry, taken from the IBM design in the last part of that sentence was
The IBM design was based on that
that old darkness timesharing idea
But they did a much, IBM in their LPARs did a much better job of encapsulating the different processes so they couldn't bump into each other.
So that, for example, you could have a time shared
their time sharing had much better firewalls between the different processes.
And it was considered kind of weird and IBM-ish the way most things are.
And the LPAR has kind of sat there.
And at that point unix
took a more open approach to the processes
that turned out to be a bad idea because they can pollute each other.
And so what happened then
is we've gone through in Unix lifecycle, I don't know if people have dealt with this, but there were jails.
And BSV had jails and then there were Charuts and there were all these different ways to try and keep people separate. And none of them worked all that well because malicious people could always get through it.
And then finally, the thing that became Docker, what's the TLA for it?
I can't think of the command line
CFXC…
Anyway, the most modern version of Pearl
separate process separation.
that allows you to create the containers
Lady.
that are isolated. Finally, they went back to the way IBM does it
And the difference now is that everyone else tried to separate processes.
a char root, you charoot a process.
The newer system takes the process group, which is a higher level structure that the processes live within, and it isolates you there
And so it's harder, it's amazingly harder for the process to get out of its container.
and inspect other parts of the system or pollute them
maliciously or accidentally.
So it's kind of funny that that whole design
that went into Unix.
fat with no one but IBM really using their version
And then IBM, people finally said, but wait, what are we doing wrong? And they went back to IBMs.
Okay.
And that's where the whole, someone remind me, I can even senile.
there's a three-letter command that starts up the containers and nobody remembers, nobody uses it because Docker is a wrapper around it.
LX.
Alexi.
LXC, thank you.
I was thinking the wrong literacy. I wrote the C, I put it at the wrong end of it. If you look at the way LXC is laid out.
you define everything at the process group
And then you create your first process.
And because the first process begins life inside of its box.
it has no way to get out.
And that uses c group file system, right?
And…
Yes, I think.
I haven't looked at the guts of it in a while. It did. I don't know if it still does.
Yeah, the newest version only allows CGroup V2 now so
They're iterating on it.
Yeah.
But again, that it
The distinction is that the
the thing that is locked in a box starts at the process group level.
And then you install your first process into the group
So the process starts out correct.
with a charit, you take your process and then you charoot
But the environment, the data structures that describe the process, all of its metadata
is left outside the true root.
And that's how people escape it.
Thank you.
malicious actors can go back to their process group and find things about themselves that they weren't supposed to know.
and the
That's one thing that's gotten Linux a lot more secure.
the LXC organization is at a higher level of data structure.
I think I got the worst of these done with.
huge bandwidth.
If anyone finds any more typos on these, please just tell me.
I'll be happy to fix them.
But one other announcement that was pointed out to me, it's actually
in reference to next month, the new log meeting
Come on.
because of the conflict with the conflict
Oh, my God.
the holidays.
is moved to i believe it's
Frigging move Christmas. My god, man.
I know. I believe the new log is
move to the 17th of December so but it
Okay.
it is marked that way on the calendar, or at least it will be.
So, uh.
That's next month. This month's new lug is still on.
you like.
Thank you.
Something like that.
Thank you.
Well, is there anything else that we need, say, before wakes up to say thank you very, very much to Steven? This was a real
Auteur de force of…
Thank you. I don't know about that, but…
If anyone gave enough of a damn about this.
It will be at speakerdeck.com slash Lembark.
in about
Oh, come on. The one thing I passionately hate about this thing is they don't have an upload button right where you can see it.
And I waste so much frigging time
We'll also have…
remembering. Oh, there it is, uploaded deck. There we go.
archive them.
Um.
I would give a call.
We'll also have the reference shown in the archive that we keep on the home page that will show
your talk tonight and the slide deck.
Thank you.
And the recording that
Okay.
Oh, that will all be in the archive.
That's there as well.
And it'll be on YouTube next week.
Yeah, you too.
Oh, God.
All right.
Nobody wants to listen to me but
Okay. Anything else we need to cover?
Okay.
Thank you all very much for coming tonight. Thanks. A big thanks to Steve. Great talk.
Okay.
And the bookend.
This has been the november
Lugby.
Okay.
today's date, Thursday.
Our speaker.
Okay.
with Stephen Park. He was talking to us about
B and L.
Okay.
No, it's not the other BD.
Gary, you're cutting out.
Thank you.
It's masochism from BSD, but it's how to use GitOps and type
Ooh.
to parse command lines and
Limes.
Good point. Anyone, publish date, category?
Technology. Give me two seconds and I'll be able to paste a um
The link for this if anyone real
Pinned.
Yeah, start that one.
via URL. If you want to give me a video URL, I can put this here
Browse and select so
um
Okay, kill recording.
Thank you.
There we go.
Get the current version of this.
There we go.
Oh, God.