Evening, folks welcome to the January Saint Louis Unage group main meeting.
We have 2 presentations tonight so we'll get to the main here in a little bit.
But first our long time, Cremudgeon, emeritus, Stan rock, art.
Well tell us more about his history and perspective of unix versus the public perception, and I will turn the floor over to you
Yes, sir, odd and clear.
Yeah.
Thank you, Lee. Am I coming through? Okay, okay, this. This is a little presentation I put together.
It's base. It's a I had developed a handout, and then I thought, Gee, that's a lot of good material for new users.
And so I thought I would put this together, and it's a we haven't recorded it before.
But it's essentially it's it's for newcomers.
I run a newcomer. New Linux users group here in St.
Louis. It's under the St. Louis Unix users group umbrella.
I have a I started out as a beginner while back I did.
I started out, the hobbyist in back in 19 seventies, while on active duty with the Army Reserve, and I eventually got that as my career.
And then I retired from there about 1993.
I did contact work. Thereafter. I've since received Linux, plus certification security plus through company to you, and that was a number of years back, so that I'm grandfathered into that.
But I don't do any certifications anymore.
I'm I'm retired, and and do stuff for the fun of it, and like learning.
But there's more about me on the the link to the homepage.
But the whole reason for this thing is to to throw this at recovering Microsoft.
Victims. And essentially, I thought, got together with a number of things.
Essentially, there's 9 specific reasons of of why to use Linux and one of the big advantages is free, you know, and that's always good and lots of.
And there's lots of free, open source application stuff that run on Linux and there's no limit of the number of installations that you can do.
Okay, that besides the the advantages, there are some chances or some negative things, it still takes time and effort to install that's that's a cost.
It's not money, and the public has a they're wary, and they don't believe that you can actually have good software if you're giving it away.
And it's and as Carl Fish always says, it's hard to sell free software, and it really is.
But then bottom thing there is cost is the least important reason, and people don't realize that.
Yes. So something about cost. Liberty, Microsoft windows or apple.
They are, they are the main alternatives, and but this is an event, an alternative to those 2.
Those are proprietary things. If you don't have those with lunch, you avoid the upgrade, the the lock in the treadmill.
If they put you in, you know, and there are a lot of restrictions licensing restrictions of what goes on there, and the Gpl license and other S open source licenses are really good advantages, and you have the local control of the changes that you have on your system.
Yeah, correspondingly with that liberty. There are some challenges.
Freedom is not necessarily an advantage, because freedom can sometimes seem like a free fall.
And sometimes democracy, because a little unruly and a lot of freedom can be a problem and the big thing is a large number of choices which often confuse a lot of people. Alright.
That's
One thing there an advantage to using Linux is, it's it's puts the fun back into computing.
For one thing, I when I first started using Linux, I thought, Wow, I loaded it up, and I'd been, yeah, look to me like a superdoss at the time back then and version 5 at Dawson. So 6.
And so on. They had, like 98 98 commands at the command line, and that's files as you go around.
And and when I jumped into to unix it, the Linux, there's like 3 or 400 of these things.
I thought that this thing never improves, I would be extremely happy.
Okay, continuing with the Linux it's a it's a full bone blown system.
And so you have the interoperability between machines. You know.
It's a it's a true clone of a networking operating system.
It's not some toy system where they later on, they thought, gee, what's that Internet thing we may have to figure out how to use that?
And so going along with the Linux. It it's been developed with with Posix standards.
So is trying to emulate or meet those standards.
And so you're gonna learn real it open standards as you as you go with you.
If you learn the the full course. So of of Linux, of course there are.
The challenge challenges the it's unfamiliar environment.
It's, it's different. And different is scary to a lot of people.
It's hard for people to change, and it's a constant changing.
It's it's a moving train. This thing has not slowed down one bit since it's started, and there's always more to learn, and that is something else.
But sometimes this gets to be difficult
I'm having a little bit of trouble with navigation here.
Of course there's a the interface is a big advantage.
The X windows system. Open standards, that's a graphical user interface.
And it's mature and it's it. There's that standard.
And it was developed by Mit, and and it took a while for Linux to get it loaded in there, and I was kind of slow to adopting it, because I had never used something at that level before, or at least on my own, and I was used to command line Linux, and then I off
Course I had Microsoft windows, cruddy windows, system and
And it was a learning curve for me. The the graphical Linux Interface style choices there's there's a number of them.
There's no Kde, and you know about 20 others.
I'm I'm enjoyed traitsing through all of those, and they were also.
There are new designs for tablets and phones that that Linux has.
Then there's also the tr, the traditional drop down menus system that are available.
And so it you can. You can have a Linux system that looks like a Microsoft windows, a 95 or window, 7 or windows.
1011, whatever they could. It can be made to look the same way if if you really want to and of course it's it's all built on the actually behind the scenes.
This all command, line, interface, because the the Guis are actually, they are just built on top of that, and the Gui ability is very good, and the command line is not required, and some people think that Linux requires a command line.
But it doesn't. The Gui capability is not common knowledge, and it's not required.
Yeah, I have beginners that use the the graphical interface.
I recommend the Linux Mint Mattee, because it looks a lot like windows.
95, and windows 7 is very familiar with that traditional pop-up.
Start menu kind of thing when menus and people can adjust to that.
But it's they don't know about the the Google capabilities.
So that keeps some people away, or reluctant to try it out, and unfortunately there are a large number of choices or fortunately it depends on how you look at it.
I mean there are in the in the past couple of years there's been about 920 different distros.
Linux distributions that have different interfaces. And currently, there is about, oh, probably a 200 that are active.
If you go to distro watch.com. They currently list the top 100 systems, and they're they're all a little bit different.
It's something to to deal with.
Security, you know as well as Bruce Schneider says, security is not a product. Is the process.
The advantages in in Linux is. There's no Microsoft viruses.
That is a huge advantage, because they're probably the last time I I looked they were saying that they were like 60,000 new Linux or not Linux Microsoft viruses.
Every month 60,000 of the darn things that's and then they they just keep constantly changing, and it's difficult to target Linux.
Well, for one thing, it's a low. It's not the low hanging free. It's it's it's, you know.
It's like, it's less than less than 5% less than maybe less than 3% of of the machines out there running it.
So the the malware writers, the the bad actors.
They're not right writing something for something that's that.
In that small environment they are looking for the big guys. They wanna catch Microsoft and apple stuff and it's got good out of the box default configurations and a lot of the distros now are pretty good for for new users.
On a stable uptimes has been measured in years for the longest time.
I mean, there are people out there that it used to be.
The thing was to see how long you could run Linux without rebooting it.
I never. People have had it up to years that I know of.
I think, hey, nobody bothers with that anymore. That was always an advantage.
Of course there's there's challenges. Nothing is perfect, you know.
Linux is not immune, for from a malware there are other malware issues besides viruses.
There are! Oh, worms and things like that that that can be problematic.
And it is a different environment and security. Wise. And it's just because it's open source does not guarantee that it's secure.
I mean, the code is available to be looked at. But people don't always look at the stuff for changes, and sometimes they look at stuff and don't realize what implications it has sometimes say it's they just don't understand.
Or or they're not really, honestly, the most of the developers are not thinking about doing writing something securely.
They're thinking about writing it and getting out the door.
So that's that can be a problem. However, when things are are or spotted.
This. It's easy to fix the code, or relatively easy for the developers
Hardware, some advantages to Linux are well.
Some of the versions actually are work with our designed to work for or for the with the older hardware there are, there are Linux distributions out there that they say we work with old hardware, and they'll I've got machines.
I hate I'll I'll admit I've got a machine or 2 they're they're running with the, Re, you know, machines that are more than 10 years old, and it work fine as far as I'm that's gonna do what I want.
And behind the scenes, behind the even, behind the graphical stuff.
It. It's just all works pretty much the same. And Linux performs well with less RAM than than Microsoft windows or or the apples.
Android
Things. I run no, or android. Well, you know, whatever.
Everyone is a version. Analytics
Yeah. Yeah. Whatever. That's you know this choice for the innovative development that's, you know.
They moved Linux over to that hardware, you know, and but there are still challenges with all this rarely pre-installed.
I'm I'm addressing mostly laptop or workstation or desktop type and personnel.
I, yeah, I just kind of ignore the the the smartphone stuff because it's it's honestly, it's too much technology for me.
Hi! I'm lucky if I can move stuff from from my desktop to my smartphone, and I've failed to do that.
But it's always a challenge, because I honestly think that that the people that are are that right for these app these android machine phones and and and the so forth.
I think they have to have what they call a frontal lobotomy before they start writing stuff, because it's so much a a different environment for me.
It's just it's crazy. And another difficulty hardware.
Wise is the newest. Hardware is not yet supported.
Cause a lot of the developers are not developers. A lot of the manufacturing is they don't actually write the drivers that are necessary to run the the hardware to to match up with Linux.
So a lot of the drivers actually have to be created by people that reverse engineer, the the the hardware, and they they create the drivers.
So if you buy a brand new machine and try to run Linux on it, you may run into a problem where it just it won't even see the the Wi-fi card or won't see the the Monitor. And sometimes they just don't work with that stuff that way.
Another problem. Recently, in last couple of years is the Bios and the Gtp.
Tpt partitioning, and the Uef issues. It's almost now.
It's almost like it. It was designed. There are allegations that it was designed to keep people from loading Linux on hardware.
I almost think that's true, because sometimes it's very difficult, and I I have only done it with older hardware and loading Linux and the newer the hardware is.
I often run into these uef buy a bios issues.
And I'm learning more about it. But it can be a problem.
And it's challenge now. Installation, as I was saying, is mostly it's easy to install.
That's an advantage. And it's easily you can download the Iso packages.
They're they're out there somebody has a distribution.
They generally make it available, or have mirror sites, for you can pull it down.
And so, and they when you have it down, a lot of the distros anymore, the isos, they they all run what they call a live live in, it's boot up. It's a live CD or live DVD, they call it.
And you the advantage to that is, you can test the hardware without actually modifying your machine without making changes to the hard drive.
Well, most users out there have never installed an operating system, you know.
They. The machine they bought came with Microsoft on it, or came with the apple, you know, and burning a nice of disk is not just copying the file
Programs to actually burn it to a DVD or burn it to, as we call burning it, to a USB drive.
There are. There are some advantages or, excuse me, there are some really good things that have come out in the last I'll say 4 years where it's a lot easier to to find that kind of software, but sometimes it's just downright hard to install on depends on the hardware I
In my experience I've had 2 2 machines from a manufacturer that you know I I don't remember exactly which.
One dell, or it might have been a HP exact same model number.
They look the same on the outside, and if you open the box it looked like the same on the inside, but it was way difficult.
There was some some little 20 cent piece of hardware that was different that made it a little bit more difficult to install.
So it's sometimes hard to install, and there the large number of choices often confuses people.
All. Where do I start? Well, I'm prejudice, and I recommend that the new comer into the field start with Linux Mint, and and there are a number of others.
I I prefer the Mette version, and then they have the cinnamon version and Ambitu is is often recommended, and they have a marketing team, and according to Mart, they're marketing people.
They are the best distribution. Okay? But
And according to marketing, sun was the best distribution. So
Oh, yeah, oh, yeah. And but then there, there are a lot of distributions out there, and the unpolished ones.
The those installations. They give a poor impression, you know, so there are some of them that just don't work quite as well
I installed the ubuntu on a fairly new laptop.
My sympathies.
Yeah.
Well, it wasn't too iron. I I want to do run
Yeah. It's just a bloated mess when you're trying to run it
Yeah.
It's it's fine. I mean, I'm using it.
And I'm doing different things with it. I'm doing the scanning with it
Yeah, okay? And going on applications. The advantage to Linux is, there's a multitude of applications out there
There's no very so there are a lot of app.
There are a number of applications that are very good, that that be cross-platform.
They'll they'll have a version of their application.
For example, labor office they have. You can install it on on Microsoft.
You can also install it on apple, and besides, you know, it.
It started out on Linux. So that's cross platform.
There's a multitude of those choices, the applications that have extensive file format support.
I mean you know you, you know you. You look at files, and there, you know, there's Pdfs, and there's exts and and C Cvs.
And you know a 100 different app formats and practically all of them.
In some shape or form, somebody is is got something out on Linux that'll that'll work with that application there's there's costs, improvements and bug fixes to to the software and licensing allows updates via the repositories.
And so it's relatively easy to to install those updates.
But the challenges are that they ultimately turn are not exact.
Replacement. They're not, and then there's legal legal restrictions on the codex and and things like that.
Some of those file formats locating the solutions that in the past has been pretty difficult.
Still kind of difficult I've recently found a site called alternative to Alternatives, plural 2.net, and you can type in a name of an application error, and it'll give you a lot of different alternatives.
That's a very good site I recently discovered in the last year or so but there's no standard of quality that that addresses everything so that can be an issue.
Okay, and the community can be very looks. I jumped the slide.
There. Sorry. Well, anyway, the community can be very helpful, and developers are usually accessible, and 80% plus of kernel developers are now paid.
It used to be the they'd say it said, Well, it's everything, in Linux is written by somebody that's in living in their mother's basement, and they haven't got a job yet.
So they got nothing better to do. Well, 80% more than 80% of the of the kernel developers now are are now paid by some corporation, and that's just a 80 plus s is a number from that I picked up searching the Internet from that was given out and in 2014 and 15 and
I've looked for that. The statistics on that, and I have yet to be able to find any any increase beyond that but it's probably higher.
We actually had a one of our speakers a number of years ago, came the Indian trails, and he talked about how Ibm was supporting changes.
And it's interesting.
Over the on Red hat. Now they care
Well, yeah, yeah, I see a little bit different now. Challeng challenges cultural differences in terminology, terminology.
The the typical new user. I'm talking to somebody is not a a geek.
Terminology is different. Yeah, it is. I started using lever office database and the documentation wasn't that great?
And what I went to do is, I went to library and got a graphical book on how to use Microsoft access.
I think that's their date, the name of their database thing.
And graphically. Page by page. They, Linux Labor office database look pretty much the same as the Microsoft thing.
They had 5 different terminologies, and that was the only difference I could see in the book, and that that helped me a lot.
They actually that book was better than the documentation that they had on the labor office at the time.
So, and determinology can be a problem. It would have been a problem for me.
Commonly there's much mean, offensive rude, and thoroughly behaviors on behaviors on the on the forums, and so forth, and sometimes, if a new person asks a question, you know that somebody will make fun of them, or or start talking about their hairitage and whether or not
They're there. Eyes are cross, or things like that, or or call names, and they may not get any help at all.
And so the big challenge is to remain friendly to the newcomers.
Now that's in the community, I think. No, I, I added, just recently added
2 other slides, and talking about this is just some advance that that really are not totally relevant, but they kind of give an idea of of things.
Some things that that that I remember running across in the in the years.
One item in in in 2,003, the department of this actually studied internally how it was using free, open source software.
Hmm.
Now I I I was in the military up for 20 plus years, and then when I left, it was forbidden for us to use free, open source software we had to purchase soft commercial off the soft stuff.
Well, guess what in the network center we just ignored that.
Because guess what? The door was locked, and most of the guys, that came in there didn't know what we were doing anyway, you know, have in the managers, you know.
Most of them, hey? I had no clue. I worked for a major one time, and in his his concept of you working was your fingers are always moving on the keyboard.
That was his concept of view working so. Oh, well, this is just the way it was with that ironic across, and I was looking for it.
I wanted to to. Somewhere on my massive hard drive. I actually have a presentation.
It was generated by the United Nations back in 2,000.
And some, and I couldn't find out the exact year.
So I just say, before 2,000, they in the United Nations was actually promoting to to Third World countries to use Linux.
No, that's interesting. Amy Warhol. 2,004.
She did a price presentation at one of the conferences, and I have a single shot of one of the slides that she has, and then that she predict she says, that she predicted the Us.
Would be the last nation to adopt Linux on the desktop.
That was the prediction she made, and in 2,004 she was an economist and was always often recognized for her.
Her take on what was going on in the It community and along the along the way the legislation lobby lobbyist guys, I call him.
Weenies. Okay, they'd like to protect that dead dinosaur technology because they have the money.
Anyway, and of course, then, the unknown is all the unknown.
New stuff is always scary, and then what did you really need?
From that time. Okay, my predictions are nearly always wrong, so I can't really predict them.
But the question is always, what do you really need? And I I I'm looking for. Who?
Where I got this quote, time solves all problems and proprietary software is a problem.
There was a I'm looking. I've got it written down somewhere.
Okay.
Who who wrote that? But that was a excellent observation.
Proprietary software is a problem. Things in the cloud will migrate to Linux, and then it may not matter.
That was another quote. I think now, if you stop and think about it, hold this stuff.
This is up in the cloud now with aws, and add your and and and the 12 other things are up there.
Miguel actually internally has a version of Linux.
They use to create their azure garbage, or excuse me, or whatever they're
Bottom line, the Internet and plus the Internet of other people's things.
IoT the Internet of things is scary. That's the quote from John Mad.
Dog, Hall.
And I've got some other additional resources that talk about advantages and challenges.
They'll be available on the slides that I'll have posted up on on the on the Internet tomorrow or so, and I'd like to.
And and my portion here we've got other sponsored groups of St.
Louis, Linux users group the which also does a presentation they'll be coming up in about what 9 days.
And then there's also 2 other free formatted groups.
There's a slack. The St. Louis Area Computer Club.
That's kind of, for we hold a a Microsoft workshop, or excuse me, a Linux, I should say a Linux workshop, and it's not restricted to Microsoft or or Apple or Linux.
It's a people are. Try to come, and when we did it face to face, it was a lot easier to solve hardware problems.
But we are actually solve some people's problems just by talking them through what what's going on.
And have we have a number of people with experience that show up there that is helpful.
And then I also host, a a new Linux newcomer group on on me on a fourth Tuesdays of the month, and I'm done here.
Anybody have any questions about what I've thrown out here?
Oh, I'd like to point out that that I will also have a link to to a handout that comes that that that matches the the slides more or less.
The slides to be holocaust. Those 2 slides about the time in the future predictions and things like that.
But it it is, would be useful if you wanted to talk somebody into trying out, or are learning more about Linux any more questions
Hmm.
Well, Lee, I guess I'm done here.
Alright guys.
Gary, you can do the Gui Medal, my friend.
Yeah. Let's see. Welcome everybody for those of you who either, maybe joining us for the first time, or perhaps haven't been here for a while.
This is the St. Louis Unix users group. It is our monthly general meeting it's one of about 4 meetings which are kind of large group meetings that we run each month of the of those 4.
2 of them are nonstructured in the sense that there's no there's no particular presentation for those nights that one of those is slack.
The St. Louis Area Computer Club, the other one is the newcomers group.
The distinction is slack. Yeah, we pretty well start out talking about hardware.
And you know how things work and just people bring in or in the age of Covid show online.
You know what they've got, and talk about what things the hardware could and couldn't do, and but it's just kind of a chat session between the folks who are there.
The Newcomers group is oriented towards, as the name says, newcomers in the sense of whoever considers themselves the the most newcomer of the group, gets to say, Hey, I've got a question, and can you help me with this, and and basically the group tries to answer
The whatever simple questions people come up with, and as the evening goes on they tend to get more complicated.
But and of course we welcome more senior people to show up, because they're the ones who can prescribe provide the answers.
So the other 2 meetings, or this one general meeting and the St.
Louis Log, Linux users group. Both of those have presentations involved the general meet.
Tonight, we have actually 2 presentations. As Lee said, you're in the Gui middle right now.
This is a cookie type meeting. The front end is the basic tutorial that stand.
Just did. The back end is a main presentation kind of a more advanced technical presentation.
And that's what we're going to have in a few minutes.
And here in the middle we do announcements, and General Q.
And a so it's a free group. There's no membership fees.
And it fact that leads into the next little item, and that is, if you show up to 2 meetings in the calendar year, you can vote and voting is what happens next month.
We've got a 5 man board of directories and 5 person board of directors, and there's one or 2 seats.
They're open next month. If anybody's interested in running talk to Stan or myself, and we'll get you on the ballot and let you know what all goes on.
Anybody who wants to vote. Basically, we take votes electronically.
Oops!
I think we all scare
We'll, you know, get the votes counted here, mice!
There we go! He's back
Can you hear me still? Okay, the deadline
Yeah. Something's flaky happened to you, but
You're back
Okay, the the devil voting next month. Yeah. I believe it's the the night of the Saint Louis lug meeting, isn't it, Sam?
Actually any, any vote that's received. Prior to this, steercom, counting the votes in the end of the month
So.
Okay, okay, alright. So anyway, you're you're certainly encouraged to vote even more encouraged to run for office and if you don't feel like running for office, I just like to offer right now a big thank you to everyone who helps put this group.
Together, Stan, who just did the presentation. I can't tell you how much Workstand does of these are act Secretary and executive vice President, Lee is the treasurer of the group, and has been keeping us financially solvent and technically functioning for some time James conroy
Is the Vice President for the lug, Sean, Shan T.
Twitter House has been doing some technical work for us and getting things done.
Film bunch is always good at keeping the the slack and the new comrades.
Meetings, running
Oops. Gary! Can you hear us, man? We can't hear you
Oh, Gary, hey, yeah, all of a sudden we can't hear Gary, but I think we ought to go into
It's the whole thing of a lot of fingers make for life to do anything big.
Just a little bit of help is really appreciated, and I forgot to mention Dave he was here event to go online.
He's one of our board of directors. So, anyway, with that, that's takes care of the more general announcements.
Next week is the St. Louis meeting, St. Louis, Linux, User Group meeting.
The topic is Empire star killer, I believe.
Go ahead!
There's some question about the exact topic on the title, but it it it uses, I think, one version uses partial.
Another version uses Java 3 or something like that, and it's not quite that clear in my mind as as exactly the details of what of this?
We're going to be looking at, but it is some kind of additional security package.
So well, we've we've we've kind of jury rigged an abstract out there on on the home page for the St.
Louis Linux users group, but if we get a further clarification, that'll be the fixed to maybe maybe not.
But anyway, it it it more likely be an interesting presentation, regardless
So, so by all means come back next week, and we'll have yet more security topics that next Thursday night week from tomorrow.
So for those of you who aren't familiar with it.
Besides having our stuff out on meet up, we maintain our own calendar, slug, org, slash, calendar.
Not only is our stuff our meetings on that calendar, but we try and put.
We used to put meetings have happened here in St.
Louis, and of course, in the age of Covid, with so much stuff online.
Now we just try and put any technical meeting that you can get to on line or physically, here in St.
Louis, if it looks like something that would entertain or inform our usual membership.
And there's our discuss email group, basically you can.
It's a just a a a mailing list. You can continue conversations of stuff we talked about tonight, or pose new questions, comments, and it's it's not a very high volume.
Group but then every now and then somebody will post something really interesting and cause a little flurry of activity.
So you're welcome to join that. Let's see.
Oh, next month! Tony, funny that you're faced should come up right now on the screen, is it?
No no Pci.
Ipc version 4. Do I have that? No Pci.
Pci version 4. Yet again, another security tool. Our security standard, I guess, is more proper term term
Yeah, it's a standard
Okay, so.
I'll I'll I'll send out something soon.
We'll get the details, and that, and that'll that'll probably be one of the talks.
In February. We haven't quite locked down the date yet, but but yeah, that'll be something to look forward to soon.
Good.
So anybody have anything else. By way of announcements
The fourth Tuesday will be the the Linux newcomer group.
Fourth Tuesday of the month
Okay.
And all are welcome.
Fantastic. And let's see.
Oh, oh, oh, the the the other thing! Let's if anybody that wants to send me $10, I'll make sure your name is not on the ballot.
Hmm.
This. This goes back to one of the years when we were having physical meetings, and I happen to walk out of the room.
I came back in the room and found out. I've been elected
You were dough enough to walk out of the room. Man, what did you expect
Yeah, anyway.
And you want done a wonderful job. Sense
Well, thank you. Okay. General Q. A. At this point you can ask a question about anything.
It has a heck of a lot better chance of being answered if it has something to do with open source, compute technology.
Linux unix. So anybody got any general questions, and they'll probably be some extra time at the end also, for those who wanted to chat at the end of the meeting as well.
But anybody got any questions. They wanna pose now.
Hello. Yeah.
Everyone with a heartbeat say, Hello!
Hi
Hello!
Hello! Everyone else is dead again. What?
Hello!
Oh, you're on mute
About half a lot of them are muted. The default
Well, I notice that's what I'm thinking people can.
Well.
Hello!
You're a lot going. Get yourself and say, Hi, people
Yeah, you, like, you didn't say unmute first.
Hi
Hello!
What's up?
Hello guys.
Then say Hello.
Oh, sorry!
By the way, just maybe a good announcement to Hi, everybody!
I'm joining for the first time from little bit south of you all in Texas, and we have our own unix user group which has been meeting first 8 days and we'll have.
We have our monthly meeting on the second Thursday of every month, so I just had a link, and then anybody is welcome to join. Thanks
Me!
It.
Fantastic. Yeah. If if, when I see that link, if you don't mind, I'll I'll add it to our calendar.
And that way people can see it on our calendar and find out where to get a hold of your great
And if you've been doing this since the eighties, I think that's older than in our Linux group.
Hmm.
So cool, and it you said this was south of New Orleans.
Sam.
He's muted
Yeah.
Yeah. Sorry I was touched that up. I know this is the Dallas Fort Worth.
Oh!
You use a group and we've been doing joint meetings with the Austin people as well.
So, yeah, about 10 million or so people
Wow!
Sam, Sam, just for I tracked the attendance in meetings.
Let's see, you're last name, or at least the initial of your last name. Please
Okay. I'm in Kansas
Thank you.
Hey!
Hello, guys, I'm introducing myself. My name is Fena Teddy from Bangalore, India. Actually.
I come across this user group on the meetup. To that, I enrolled for it.
So I'm yeah.
Excellent! Welcome.
Yeah, that's great.
Welcome! Up! Welcome! Welcome!
Thank you.
It's always always good to know where where people find out.
Okay.
Okay, our meetings. It's like less. Carl says it's hard to sell free software.
Okay.
It's even harder to advertise. And the meeting sometimes. And it's just it's a lot of work and
How often we we get a lot of well, honestly, I I don't like meet up because there's a lot of problems with it, and it's yet another thing we have to deal with.
So we, as it turns out, we don't have a lot of Indians we've got a lot of chiefs.
Wandering around in our our group, and we always welcome people if they want to help out.
So if they want to help out talk to Gary, and you know you can write mail to gary@sloop.org if you want to volunteer for something he'd appreciate dealing with it, and I we'll probably be prompting gary to check his mail more often
Excellent thank you, and welcome to everybody who is new or recently returning, and with that again there'll be at the end of the meeting if people want to ask general questions. I'm sure they're welcome.
I'm gonna introduce our main speaker. Our main speaker this evening is Stephen Lombard, a long time member of our group, you no longer lives in St.
Louis. He's moved to the east coast. He's the principal at so work cars consulting
Workforce computing. That's me.
Nowhere here. What is that?
Of course, computing. And he's he was leader of the Pearl Mongers group here in St.
Louis. And so with that, he's going to talk to us tonight, and I'm just gonna turn it over to him and cause it sounds like a really interesting thing.
So. And I'm gonna say this upfront, I believe I'm correct, Stephen, you like taking questions right time. Right?
So somebody has a comp question
Sure anyone has a way keeps me from having to talk.
Alright!
That's fine. Well, let me originally what I was going to do.
I were using zoom, and they're been some questions over time about whether or not Zoom was particularly secure.
They have been problems. People had in the past looking at it where they.
There was one room. If it was yeah, try to back door things.
One of the interesting problems you have with Zoom is, it's?
And this is another issue you've got with secure and open source, which is closed is nobody really knows what's in zoom that is produced by a Chinese company, and what's interesting about that is all Chinese companies are owned by the Chinese government.
So a Chinese
Yeah, let us say, a Communist trying to use company
Well, 3 P. No, I did. They're not Taiwanese.
So anyone from mainland China, technically, is owned by the Government.
What's interesting about that is, you go back a couple of years.
You remember those loss and the the Feds here were suing apple to decipher the content of a phone, and they were eventually told to bugger off by the ports
What's interesting about that is in China. That doesn't happen.
People have to obey, because they're owned by the government, and that's true.
In Russia. That's true. In a bunch of places.
So you know one of the questions is, how can I tell what is going on?
So what I was gonna do here. And this is kind of let's see, Abradabra, people should be able to see my screen right
Yes, we see it.
Yes.
There you go. I did some. Right. Okay. Good night. Now, anyway.
So basically what we're gonna look at here, you know, I and this is, you know, Zoom is popular.
Okay.
But so is cocaine, so is there a downside to using it in tweet. Find that out
And the the real thing here is using unix in general.
But in particular, Linux, it is actually pretty easy to ask the question, What is going on under the hood?
And that matters because unix is a single kernel, and by that what happens is when you go into access shared resources.
Memory just drive, you have to ask a central service for the access that you need, and we can watch a particular process and see what it's asking for, and because that's a standard interface, you know what to look for when you say, but wait what is this doing, no it's the thing just runs around
By itself, internally, there isn't much it can do to you so normally for security problems, something has to cross that kernel boundary it has to make what's called a systems call in order to get into the inside of the shared portion of what's right and extract what it wants from
There. So the base idea behind security is, you only want legitimate access there are a lot of different ways to say this, but ultimately you want to say that people who are supposed to get things get them.
And people who are don't so what we're gonna look at today is how to tell what something is accessing.
Now I've got a couple of really trivial cases.
We'll start with, and then we'll use zoom and and see what it's doing.
We'll also be able to look at how efficient it is.
We can look at how much resource it's using.
And this is useful for any program that you're you're going to use.
So the shared resources in particular, you have to. You can look at what goes across that systems boundary.
The system calls with a utility called Estrace, and I I apologize.
I just finished hacking me, so I watch the slide sorting o oops.
Sorry the system calls for that interface process. Control is one of them.
If you want to fork, you want to create in process.
Every time you open a new window every time you start a new tab.
Every time you run a in the shell every time you type.
Ls, any one of these things creates a new process.
Maybe very short lived. But that process creation is a standard kernel call anytime you want to access something.
Our directories on unix or files. Everything on unix is a file.
The only thing you can read and write on unix is a file.
There are a lot of different types of files, but there's a standard interface for file access.
So anytime you go into a directory, I can say, wait!
This person is doing scandal. This person is, is reading from a directory file.
I can see if someone's trying to. For example, scan your file system when you're accessing network ports, opening a port, closing a port, sending data to a port is another type of system.
Call.
So everyone here that has access to Linux right now can type as trace minus help.
And you'll see something like
Back to this one. Nope.
Invalid options.
Well, well, well, you were talking about pieces of hardware I was trying to set up when people get bored with this.
I can show you what another computer looks like when it I just bought to replace the old machine that I've been using for the last 15 years.
But if you type S trace minus help
Oh, if you're on the correct machine and you type S, trace, what?
Oh, this is, I apologize. Yeah.
Even could you increase the font size? That's really small
Hmm.
Okay. One nice thing about x 11,
That's how long it takes to increase the font size I don't have to go to a manager and say, Please do this, do that to the other.
I can make the fun as big as I want, just by hitting a couple of keys, throws
Which is to say, that Unix was designed for old part like this. They can't see what the hell they're doing
Why is that more legible
Go go back up one or 2 just to to to, for those of us with old eyes
By the way, to do this, if anyone had cares, I'm making control shift in the plus the equal sign control shift equals is a plus and a control plus will increase your font size on on standard X terms.
Guess if you had to. What do decreases the size
Control minus
No.
Shift minus, of course, yeah, control minus
There you go. One important difference between Linux and windows.
Linux tends to make sense
Cool.
But as trace most things on Linux, if you haven't used them before, if you type a minus help they'll give you all sorts of garbage.
There's a great command. Line, utility. There used to be a utility called more, and the problem with more is, it would only go down a file.
You could never go back up it so stolen the guy that created the goodoo project replaced it, and of course the replacement for more is once so if you bite this to less, you can see all sorts of things that it will do.
If you're not sure how a command runs, you want to see more than just the the summary of it.
You can type manes trace. And this is actually pretty good, because it will start showing you what different kinds of things look like and how to get into them with the options mean. There are examples of how to use it in here
So the S trace utility is one of the best ways.
If you care about getting into how these things work. No, I was saved some bandwidth.
No, recurred is what I look like.
I'll stop! Wasting bandwidth.
So this all relates to secure, because to get correct access, you need to be granting the appropriate access to the things that are actually running.
So S. Trace allows you to look at what actually is happening, not what people stay is happening.
Hmm.
This is also a great thing. If you've got a program that croaks after 5 min of running without giving you an error message, and you've got to say, you know, something out to the Internet, you're accessing a group.
Gee! This thing works for 5 min, and it dies only so you're fine. You W.
What were you doing at the time, estrace it, get the last error message it got from a kernel call.
Find out what it error. It was reporting when it exited, find out with the exit status.
Once. You can get that from mysteries trivially, and if you have that kind of information in the question that you post your, I have a lot more likely to get an answer
The other thing you can start doing with S. Trace is asking if processes are efficient.
So if you run estrace, you might find I will see this in a minute with Zoom.
You can see people polling rather than doing a blocking read, and you'll a process will be looking and looking and looking in every you know, couple of hundredths of a second. It will.
It will waste CPU cycles by scanning something that isn't there.
You can find going
Is that? How it's fine? Is that how it finds memory leaks
You can find. Memory leaks with S trace by looking for breaks, because when you expand, when you access memory, when you extend the memory, the function call, there is esper sprk, and when you see those, if you watch something and it keeps allocating, more memory it isn't a
Guarantee. But it's a pretty good sign of a memory link
So the it'll show you the allocation requests.
It'll show you. File reads again. If if you see something opening and rereading the same file a 100 times, you might want to ask why
So about the simplest C program you can write is this
And if I did this right no, that's the long computer.
I did this right here we go. So this is
Not at not see for anyone here who speaks. C. That's the least amount of C.
You can write that will do anything. It doesn't do much.
It just starts and immediately exists. But it's there if someone hasn't seen it before, I can type Gcc minus out. Not on.
Not a dot c
If you're trying to track things on the disk.
By the way, Ls minus Lt. Is your friend. People, do these all these different things.
But the last thing you put will be at the top if you've got a zillion files in a directory, you do an Ls minus salt and pipe back to head.
If you learn nothing else tonight, if you learn that your life will be made much easier when you're trying to figure out what the last thing in a huge directory like Tenth is.
Hmm.
So I've got this file. Nada
Notice that on Unix you don't have little extensions.
Most of the extensions that most people put on. Most of the files on most of unix are garbage you don't need a dot. E.
X e, or dot dot p y to run something. Their joke, as a matter of fact, where I work right now, we've got a bunch of files that we're originally written as perl, and are now shell the calls perl.
So we've got a bunch of things that we can't change, that all have the wrong extensions on them.
Extensions are dumb most of the time. If you have something like Gzip managing it, great use the extension it puts on, but for executables do yourself a favor and don't bother.
But you'll notice this is an executable. It's 64 bit you probably don't care about most of this, but the important thing to notice is that Lg.
Linux is what's gonna run it. The shared object live library is what is going to run it.
Notice that this is the interpreter. Everything on Linux is interpreting pearls and
But they're also compiled rolls a compiled language.
So is Python. Shell is interpreted because every line has to be re-examined.
But the interpreter here is the Core Execution library.
It goes directly in. So if I run, Nada
Oops, and it's on my path. Okay, it doesn't do much
I can type S. Trace not up.
Hmm.
And now that it doesn't all fit on the screen, I can bite this to less
So what this is saying. Exec ve is the way the interpreter that library says, Oh, you want me to run a file.
I guess I I bet you I better load it into memory and start running it
Break is the the function that is used to examine memory by the kernel so what we're gonna do is cop the space that it's gonna take to store this thing in memory.
I have to allocate that memory which I just did.
Now I'm calling a preload. Most of this nobody cares about but
L. The live C is reading. This is reading the the executable into memory, and getting ready to run it.
We close it and protect some memory. This is something that is done in order to make sure that running code doesn't write to places in memory. It's not supposed to
At that point it finishes running, and it exists. So all we really did is tell Linux make the space to run this and immediately it just exists.
But you can see these are all the calls
Excuse me, what? What? What was the part that told it?
This is the space you can use, not somewhere else.
Break.
Right? Okay.
Brk is is, how
Oops. Well, if you're man, pages aren't up to date, I apologize
How to break in the one of the better places you can find a lot of the stuff if you just look up Linux, and the system calls, you can see what these things are stacking.
Overflow is a good place to look up these sorts of things.
If you're not sure I don't know if anyone here is seen stack, overflow before.
It is a programming resource. You can search for Linux, brk
And oops.
Oh, God, sorry about that! I need to disable that is one of the things.
If you if you actually use
A lot of security tools, lots of places don't do a very good job.
What is the break system? Call? Do stack, overflow.
If you're not used to looking this stuff up is a great place to start, because this is for programmers.
S. Break is used to manage where you're the end of your virtual memory is so that, break and S break.
This explains how they work and and what they do
But then we do a memory map is how you read things.
That use people are when you look at something like K. Andr.
Really, old copies of C. Programming. You'll see everyone does opens and reads and writes, and the more modern ways you memory is called memory mapped.
I, O, and if everyone wants to stay awake we won't get started with that.
Now. But it's a faster way to do things, and it's the way everything everyone does.
It. So you open something at a at a particular offset.
You start reading it, and there is a much going on, so it exits almost immediately but that's what it looks like.
That's the least amount of work you can do to get something running.
And then, of course, life wouldn't be complete if we didn't create.
Hello, world, right?
So looking at that one
A lot of this is gonna look pretty familiar. Exec, ve is how you run a program.
That says to Linux, Here's a file. I want it stuffed into memory and executing and we're going to memory map.
And we're gonna access the preloader. And it's gonna go down.
Ld dot cash live c dot 6. Now, what's gonna happen here, though?
You'll see. We get down a little bit further. It reads the source code.
It does a break, and then it does a right which shows up on the screen.
Hallelujah! The right returns the number of bytes written, not characters.
C. Does not know about Utf 8 or anything else. All it knows is bytes.
If these were Utf 8 characters or Utf, 16 or 32 C. Doesn't know what it's doing.
It's it's up to whatever displays it.
To understand this. So the right is just stuffing these bytes out.
It just happens to be 15 Byte at that point.
It adds it because it's done
But this is the same basic thing we saw a minute ago load this into memory, find a place to start the execution, protect the stack so that it doesn't get stepped on.
That's this limit, the stack to avoid. Stack overruns.
But if you look at a lot of the security problems you have in life, they're caused by people finding ways to get into places in memory on the machine that they don't belong.
So limiting the stack, so you don't get what's called a stack overflow that runs above where it's supposed to go into it, into an extra section of memory that is one of the ways you you armor.
The code. A lot of this improtect and stack is part of how you compile the kernel.
What you can actually, when you do things called hardening. The Colonel, you make sure that every process does this for itself when it's run.
It's. It's part of the normal procedure the kernel is going to use to to make you available or execution.
So anyway, we do a break and write this and that. Not too exciting. But does this basic process make sense to people loaded in memory, allocate the memory
You know that there's something out there that has to know how to run it, and that's that live C dot dot 6.
That is the place to go to get this run. Is this making sense so far
It makes sense to me. Thank you, Stan.
Is anyone awake?
I'm quick comment. 5 min ago, Stephen, use the term stack.
Overflow, referring to April website, where you could find documentation just now, he's the term stack overflow.
Describing what happens if you exceed that address space that he just showed, and
And that is why the name was chosen because a stack overflow is when you have too much stuff in memory, and the stack overflow website was when your brain does a stack overflow.
This is somebody else you can ask
It's actually a joke.
Good Explanation
Is. Actually, if you look here, you'll see if people can read it.
The quick thing is, you'll see this picture right here.
Your stack grows down the heap grows up and stack overflows.
When you can push the stack into the desk and the text area text is the running code stack, eat data and text our names that were taken from Ibm mainframes a long long time ago.
By the way, Stan mentioned, you know, running on old hardware.
One of them. Main reasons that Ibm bought red hat is that now most of the people using most of the mainframes, run Linux on them, using red hat in Lpars on mainframes.
So, the the core operating system, the the stuff that most people are doing with a lot of these V, a lot of these huge mainframes.
They've got is running Red Hat
So, where there's an irony in that. But the so going back to the talk, another way, you could do the same thing.
I could print hell. A world is using a shell
So we guess, what does it look like to try and just print?
Hello! World from a shell, and I have here
Oops! Did I lose it? Oh, well.
Apologies for that. So at that point I can say S.
3 I mean
Yes.
You can see what's going on here. Right? What's interesting is
This shebang, these 2 Byte pound bang!
Scholarship. Bang! The bank comes from the fact that in the military they call exclamation points banks, because they look like a cudgel.
You could hit somebody with
The ship bang lines, if that's bash if it's pearl if it's python, if it's rust, if it's whatever this is, something that the Colonel knows it about those 2 Byte, tell the Linux Kernel what I try to run
this start by finding this thing. When you find it, feed this whole file to it as the input
Does that make sense to people
It does to me, Stan. I think it does
Anybody? Is anyone awake?
We're all awake
Okay.
Oh, trust me, I'm very much awake.
Yes, yes.
Or we're all awake, but we're muted good
I unmute yourself. It's like, Oh, okay.
Oh, okay.
You know.
So if I do, an s trace of this
Yes, what? We're at the same place. We've always been exact.
Ve is how you tell the Linux kernel, hey?
I wanna run that and the file, the file is going to be run.
Is this, and we do a break to check that. There's memory, and we memory.
We do an M map to load it, and we do the preload and the open live.
Read line. Now, here's something interesting. We haven't seen this before.
This is a shared object library that is used by that executable.
So once we've loaded bash Vash is waiting.
Library line is something I need live. T. Info is how Bash gets information about the terminal, so it knows how to print things.
Now we're@livec.so, and we're actually running things.
I have forgotten what this does. It's been a long day.
But notice, we're doing the memory map we're doing, protect, read, protect, right now we're memory protecting a few more places.
But we're still protecting them. We're still limiting the stack.
We're opening the the Tty for output. Not notice
This is Stan, the stamp
Steve, would you go back? Would you go back up about 10 lines
See.
Hey!
Yeah, pick a line
Where the where the mallocs were
Did I see? Was that
You'll never see a mallet. Malloc is not a system call.
It's a stand. It's a C. Standard library.
Okay. Yeah.
Call what you'll see what Mallet does if it needs to.
Is, it calls Br. To sbre to adjust the virtual memory, allocation
No, it. It looked like it was allocating. 3 16 gig chunks and 2 or 16.
Yeah. 16 Meg chunks and 2 8 Meg chunks
Yeah. No one ever said Bash was lightweight just fast.
Give me a line, number
Line number
There, there it is, there it is 16, 3, 84 times 3 81 92 times have a project 58 58 to 63
Yeah.
Okay. Yes. Well, no, that's unprotect. That's not an allocation.
That's taking one to. That's taking some region of memory and making them read only
Yeah. But that's that's
So I took 16, 3D. 4 Byte and and map them is read only
Right, but that's memory that it's using. Right?
Okay, yeah. That was the question.
Got it
Right at these offsets. Those are virtual offsets, I mean the virtual memory
I know you understand this lead, but the whole point of virtual memory going back to Vms.
And and risk is was that
And and one of the things that Unix was a very early place.
You could get in the days. When heavy duty machines had 64 kB of memory on the space that you use, that you take care of.
Mapped can be larger than physical memory, and what I can do is I can say, well, there's there's hardware that does this nowadays.
Every time you try and access a piece of memory, I'm going to take your virtual address, and I'm going to map it to a physical address so that I know what you're actually doing.
That slows things down. That's why mainframes are still in use today, because they're damned fast.
But the virtual memory approach is much more flexible because I can.
I can take a space that is larger than what fits into memory, and pre-map it.
If it doesn't fit, I can stuff it into swap, or I can unmap it and call it up later.
So these huge numbers here you're right, are, you know.
Okay.
I don't have that much memory on on this notebook, but the virtual space has a very high offset, so that it allows me to grow the stack later.
If I wanted to. If you compile something 64 bit clean, you'll see insanely large numbers for the offsets of the stack and the heat, and that's just the way of keeping them from ever colliding
But this was protecting 16, 3, 84 Byte, and add a couple of memory.
Okay. I think those those are bytes. They're not pages
We could look it up. But yeah, it's not that I've allocated that much. It's that's the virtual address, and only some of that is physically mapped
And again. Here we took the Pr limit. I took the stack.
I, and set it to our limit, infinity open to teaching.
Y. Now, what are the interesting things that bash provides?
Wha? Wha? Wha? What's a pr
You is a variable called Randy
I'm sorry. What
What's a Pr. Though? What is a Pr limit? And it's infinity. That's awfully big
Yeah, I know it. Basically says our Lim. Max, is that that is a a symbol for the upper limit
It just says I don't. I'm not. I'm not.
I'm intentionally not limiting myself. My current limit is that which is 81 90, which is 8. Meg
So I'm I'm currently limited to an 8 MB stack
Now the are limit. 64. Infinity is probably all Fs, you know, minus one unsigned. I I don't know how it's defined
But that's just a symbol, so that you don't have to guess at a random number.
There's there's a number that means when it grow as far as it can
When I start this. I don't know how much swap I might have, so I'm just gonna keep allocating it if it doesn't fit into memory.
I'm gonna stick it in swap
Oh, get random! Is call you like what you know. Why do I care about a random number when I start bash?
There's a variable in bash called dollar uppercase.
The word random, which every time you access it gives you a different number between 0 and 6, 5, 5, 3, 5, or 16, 3, 85 forgot wage.
So this was seating the random number
Then we do a break. We adjust memory.
What I was looking for here. If you look at this signals, that's Seth setting signal handlers.
Now we gotta say, Hey, wait a minute.
This is where I. This is my working directory. Do I have access to it?
Do? I have access to home? Do I have access to this?
God damn!
Each of these has to be validated to make sure that I'm allowed to look at it and see what it is.
So this stack ends up, causing a bunch more of them.
Oh, this is my working directory. Okay, I know I I'm allowed to be here.
But then it has to check each of these to say, Oh, gee!
I want to do a list of the contents of this
Am, I allowed, and it turns out that I am
I/O. Ctl, or how you control terminals.
I'm not gonna go into all of this, but you can see here, it's always my bash. R. C.
File, because the first thing you do when you start up, bash is, you take the bash.
If there's a bashy, and V, you take that, and you're running.
So this is saying, Okay, this is, let's reading my, it's sourcing my Se.
Bash, rc, file. So this is opening. Bash R.
C. Goes through the whole, the whole rest of it. Now I've got.
Say, Hello! It's open. It's a read-only file.
It will not modify itself on the disk I'm gonna run it.
And somewhere down here we've got read. Bin bash you at that already.
35 Byte of it, I guess. What I do.
I write? Hello, world.
And I didn't have any leading new lines. So this only wrote 14 Byte
But this is what it takes to start up a program. If you, if you care about the security you could notice, for example, you see how this is doing all these stats to find whether or not I'm supposed to be able to access this directory, what if one of these directories was home somebody else on the machine
What if this was a directory that had nothing to do with where I was running?
That would be a pretty good indication, something bad's going on if this was trying to read from bar mail right now, and I'm not doing anything with mail.
I'm just pricking. Hello, world! You could stop and ask, why would this thing be accessing that particular directory?
A lot of the times when people have found really nefarious programs, they've run something like estrace.
And you start gripping it for directories. Just find all the slashes, and you start asking, What is it? Looking at?
What is it trying to find? And no, it's not fun.
But if we know that we're starting things, I could take this, and I could say, S trace this and just gret new fsat
Well, these are the things that I'm th. This program is trying to access
Now, these look perfectly normal. This is, it's it's reasonable to try and act, to ask.
Am I supposed to be accessing the the place where I am?
Maybe I am. I'm not supposed to be even in this directory running anything. Who knows
But this is how you can tell if something flaky is going on
Now I might want to say, let's do a listing.
This requires handling a fort, because the Ls program has to be executed.
So now echo is built into bash, but Ls is A is a whole new thing, and I I apologize.
I deleted all the stuff I set up for this, so
Hi! Damn it! So it vial stands for Vi like emacs.
It's it's a copy of micro Emacs that has Standard Vi commands on it, and it's interesting.
Everyone's got their own editor. There's you know what Stan was saying before about the different Linux distributions most of the time.
Most of the people in most of the places don't care that much about the Linux is pro.
A number of years ago, the the Linux users group in New York City.
Somebody want a paintball tournament for like 50 people, or something like that.
The lottery and we had to divide up on teams and people couldn't divide up on distros.
They couldn't divide up on parties they couldn't divide up on anything.
Finally it came down to emacs versus Vi.
But okay, so this does an Ls minus L of 2.
I I have lost it!
And here we see exec ve
And we see this thing running. It's opening it.
's E. You see the same collection of things, but now you see a lot more.
Well to make this fit. Gary. I apologize
How people are going blind. But notice that now there's a whole lot more going on here, because doing that listing
Requires getting the Ls program. Now, this everyone's seen before.
This is checking that I, the Directory. I'm running for is a place I'm allowed to be.
But now we're doing new fsat of opt being Als.
Well, wait. Why would this be happening? Can anyone think of it?
And the people that really know just wait it out
What do you notice about all of these
Somebody. Come on, man.
Would that be with that? Would that be your would that be your path?
Stan. I just said, the people who already know this stuff and shut up.
Yes.
What this is doing is making my path opt in ls, it's looking for the Ls program.
L. O, vm, 8, user, local spin. Use local PIN user PIN.
I don't put user local on my path. Normally, I I did it here to show this.
Spinls. Vin Ls finally came up. Okay, that tells me that Ben Ls is the program that I'm running, because it just said Ls, this thing has to go find it.
That's one thing about your path. If the path gets hideously long, you can spend a lot of time doing this every time you run any program
Hey? You're right!
So now in kernel land. Ls finally figured out, okay, yeah, I'm allowed to access this.
So now it went in. It did this. We created a new program.
Okay.
It cloned the work. This space, which is what? What forkins of calling internally and
This is not now what you notice here. This clone is starting the new process.
That's executing the Ls executable
The trick is all I see. Here is the output of Ls.
I don't actually see myself. What did Ls do to do that?
You have to give another couple of flags to S.
Trace, so that it will actually trace across sports
So if you're trying to validate something that's you know, J.
Random software. And your you're trying to find out if it works right, you're going to have to add, trace it across the phone boundaries
So now let's try. Let's try. It's tracing
I'm sorry. Say again what boundary
Fork. When you create a new process
Thank you. Thank you. Thank you.
This thing. Just showed the output of Ls. It didn't show Ls running.
It didn't show Ls opening the directory and getting all the the content
The thing is, let's just see, is trace zoom
Now I want to show this starting up. Then I'm gonna show it running. But
Okay, so we found user been Zoom, that's good.
We're running. Well, wait a minute. Here. You see this live read line
Does this look familiar to anybody
This is
Wait! Zoom has a command line interface. Why else would it use liberty?
No? Well, what it means is
Rhymes.
One of the more useful commands, and this is why extensions are usually garbage on unix the file command
User bin zoom is a shell script
That's why it looked like starting up. Bash! That's what it was doing.
If we
The zoom that I'm running is this thing
If I take a look at it.
What I want to do is export this, and then CD.
To opt zoom
And then s trace
Zoom.
Alright. Hey, Steve, on my system, user been zoom is the same link
I I believe you on mine. It's just it's a shell wrapper who knows?
So this shows me zoom is started again. It forked
If you look through all of this stuff here, you'll find somewhere.
Wait for this is creating a back is this background in a process?
Now what happens? I found this out just before the start of the talk.
You can't run 2 copies of zoom on the same machine.
It doesn't like it because it it it tries to reallocate the same things, so I can't.
I did all of the testing on this, but at home anybody here when we're done with this could type S.
Trace zoom, and you can see it running. You can look for these new Fs stats, and you can ask, well, gee!
What is it? Looking at one of the things that people were concerned about the past is that Zoom kept trying to read things that that works appropriate, that weren't strictly related to talk to somebody, and that's where I think that was one of the areas where apple them.
Off the apple sport because they were looking at things that they shouldn't have been.
Unfortunately, I can't show anybody anymore of it, because this is where it ends.
I can't run. I I don't know how to run 2 copies of Zoom on the same machine
Hmm.
Bye, I you know what I can do here
These are the files.
You know I I'm not sure entirely you know. Zoom platforms.
This is. This is Zoom opening all of the libraries that it uses in order to operate
So we have opt. Qt, so this uses qt, there's opt hardware caps.
I don't know what as well does
Looks like you can pass Tag P. And then the p of your zoom to get it as trace to attach.
They're currently running process
Yes, you can.
What actually wanted to show people how it was starting up and running.
If you want to do it that way, I can do a
Oh, the problem is, there's a zoom, tg, and opt zoom, zoom and opt zoom zoom, and opt zoom zoom and opt zoom zoom, add up zoom, zoom!
Zoom, zoom, zoom.
I only have one copy of Zoom running. So it forks multiple things to get this done.
And if you look at it here, they're not type people.
Zygote. High people gpu process type, equal utility.
So probably this is the parent process 4 75, 87, 88.
Yeah. So if I do, an S trace.
And an existing process is, I think, minus E. But I could be wrong
Minus P successful failed.
The code, no
This trace, minus P. P.
Now I have to have access to this process, which means I have to pretty much.
I have to have been the one that's worked it, or I have to be super user.
You can't randomly estimate other people's processes, because the unix Colonel will say, No, you're not supposed to look at other people's memory. We're supposed to be safe inside our own calls.
Kill, joy.
Well, could you
Yeah, now, one of the things you'll notice there's a lot going on here.
One of the things this does, which is why I wanted to show it to people, even when it's not active.
These these polls. This is looking for keystrokes.
It's looking for video information. It's it spends a lot of time.
Just checking to see if there's something out there that has to be shown on the screen, or checking to see if there's new sound information that's coming from the the microphone device or checking to see if there's a new keystroke buffer Buffer Buffered
And waiting for it. So this is the the overhead of you know.
If you look at zoom, if I run top
And I spell my own in correctly
So zoom right now is using a 104% of the CPU 114% of the CPU basically, what that means is they're multiple threads oops, red count
Process right? That up here no.
Nth. There you go!
You know I I got it.
So this, yeah, see, zoom is creating new threads
To handle. You see the threads go away. Sometimes it's over one, sometimes it's other, so I think it.
It might be using. Some of those threads are are short lived, and they're only for processing chunks of video transmitting a chunk of video transmitting each other sound
You know, this, this machine doesn't have a lot of processors, but you can see it.
It's pretty much stop all this. It's bouncing between process between 0 and one and 0 and one.
Sometimes, if you were looking at that, you've got a machine with more than a few cpus on it.
That's a sign that you should really use tax set.
It keeps something on one processor, so that it's more efficient
But anyway, so interesting to note that in the course of this meeting I've got 84 min of CPU time and I've only been wasting your time for just about an hour.
So that looks about right? Most of the time. Zoom seems to be using on the order of one CPU.
It's basically swallowing one core
What this is, how you. This is, what it's doing, and you can look if you care.
You could look at this and say, Okay, it's polling.
It's got this much time. Few texts is is waiting for a a mutex waiting for a shared resource so it's polling for readers, for example, to say, is there data in the audio input buffer?
Oh, there is audio, input, buffer. So it's not gonna block and wait for the Colonel to say, you have endpoint, because then it will be able to sample the audio and the video and the screen capture.
And the keystrokes, and also give me back information that's coming back over the next.
So all this polling is about making sure that it checks things pretty fast to see it.
Is, there's something there that has to be stuffed into the didn't speak, or the screen, or whatever
Screen, or whatever
Is that making sense
I think
I'm getting a really bad echo. So I'm probably overloading the the throughput with all this stuff on the screen
I'm muted gary
But anyway, so this is basically, if you want to, you can have something somehow.
I get back in edit mode
If you wanted to it's a useful exercise to do.
Write something like this, something that creates a pipe.
You can start to see what's going on under the wood.
And again, if you're into security, this is how security people start asking is something oddball going on is something.
Try to allocate more memory than it should. It may be try to blow up the stack if something is accessing files that make no sense.
That's a sign that it's trying to find files to send back to someone else.
How would it
Malware. Usually the the the giveaway to malware is it's trying to access places on your disk drive that it shouldn't be
How would it show that it's transferring files that it shouldn't
You would start to see it. F. New stat of a file that made no sense.
If you look
Let's see, where is a number. So let's say I'm looking at
Oh, that's why
This is all places where all these new Fs stats our places where the running program is trying to ask, Do I have act?
Is. Is this thing out there? Can I find it
Does that make sense
Yeah, sort of
This is how you ask the Colonel. Tell me about a file.
You stack something. It tells you who owns it. It's size
Yes, yes, that I understand. Yes.
Okay. So if I
If I just do this, if I look at all of the new Fs stats
These are the files that this copy is zoom. Make it easier to trace
This is the stuff that this copy of Zoom is trying to access, or this program.
Excuse me, is trying to access
So these are the things that it's looking at. Let's just say one of these was Etsy
Ssh! You know I was trying to read one of my my system.
Ssh config files
It has no reason to want to do that. That's a sign that it's trying to backdoor some data out of my machine.
Let's say this is running zoom, and it's trying to access my personal dot.
Ssh. Directory and read the config file. There is no reason in hell that Zoom wants to know what my personal Ssh.
Config file looks like it's not using it as H.
Or your or your private key
So looking at the list of files here gives you an idea whether or not it's trying to play around in places it shouldn't be
Or or steal your private key, or etc.,
Steal a private key. Now, if we look at these
So 6 times to try to find the Ls file 2 times to try to find zoom
This is the the director I'm running out of. If you look at these things, they're all reasonable
This is my working directory, and tried to. You know the system tried to verify.
None of these looks nefarious. If one of these had home limbark.
Ssh, computer it. I begin to wonder what the haze is going on here
Does that make sense
Yeah, yes, that does make sense. Yes.
And all I did here is, I said, well, let's let's look at the new Fs.
Stats, and then I said, But wait, I don't.
You know all this crap. I don't really care about it. Now.
Another give way. See what says Eno, ant no such file or directory.
Another way to do that
Is to say, what are the files that it cannot find that it's looking for
Okay. Well, opt zoom, as well, you know, user spin.
Ls user, local spin ls, these are pretty innocuous.
This is obviously looking down my path to find things. This is looking down Zoom, standard path to find things as well.
It's trying to find the has will shared object library and using the Ld library path fine again.
None of these things they couldn't find were any anything nefarious.
If you want to find, what did it get?
By the way, what's a
These are the files actually found
So.
Found Als. It found out zoom again. Nothing nefarious.
If one of these were something like Etsy Ssh, you know.
Ssh. Keys, construct to wonder what it's doing here.
Is it trying to backdoor itself into my system?
There are secure systems when you're getting into. I mean, like government secure systems.
Where they will track this stuff. They'll use things like splunk. And every file that gets opened on the machine gets watched. And if you try and access the file that you're not supposed to, someone's gonna ask you why cause you're not supposed to access files, not suppose
So you know, there's some really hardened systems we're trying to access a file outside of a certain list of directories will.
We'll set off alarms, and that's how you you track malware.
One of the things they'll do is there are things called tripwire programs where you can.
Basically, you'll scan the kernel, and I will take the kernel that's compiled for security purposes, where it automatically reports all the stuff.
And there's certain places they're they're called honeypots, where I will put, let's say, an Ssh config file that has wide open access, and if something tries to read that I know it's trying to backdoor myself.
So there are times when people will set up virtual machines, or or Linux containers that have things in them that are not properly security.
The question is, Do does any of the software running on this try to access that honeypot?
And if it does, then we know we found what some malware
But anyway, this is just lowly estrace, which is
This is just pricing what happens when you list some files
What's the term eel? What's the term?
Eno end indicative of
No entry, you know. Edge error, no entry.
Thank you.
So if you stack something.
St. Is the system call that says, tell me the information about something it gives you. The user.
Id the group id the file, protection, the size, the number of references to the to the links.
So how many links to the actual inode are present?
So it reads all that metadata out of the inode, and the the stack call for homeland bar term info.
When the running program asked, it was said, no, there doesn't exist, you know.
End is is just a standard. C header macro. So you get Eno in.
Sorry that does not exist. I don't have a term in full file in my home directory, so at that point I don't have an overriding term info.
So it's gonna ask for ec term info, which does exist.
The system call returns 0, which is success in Z
After it looked in fc, it looked in user share term info, probably because that's the term info's got a line in it.
Let's get user share term info in it. At that point it says, fine.
I found this. This is the standard terminal. So now we're looking at se tormento turt, 2, 56 color extern
So we open that for read only, and it worked. Turn, return, file number 3.
File descriptor number 3. Then you go down it that way.
Another giveaway is you look for opens that don't have.
Oh, underscore read only
You think about it. There is nothing on this machine that's got any good reason to try and access any of this stuff except in read-only mode.
I'm not asking it to modify this stuff. I'm just listening a directory now.
Ls needs to know things about the terminal, because I may have turned on the the funny colors in my listing, so it has to ask for the.
But it has no reason to write that I can't.
A way. People find things that have security holes, it is you run S.
Trace on them. And you ask, Is there something that's trying to open a thing in right mode that it shouldn't be accessing
Was this helpful at all to anybody
Okay.
Yes, I'll
Okay.
Very informative. Is there a specific code for right? I see the I understand that all our
Read, only
Oh, read. Only yeah. Oh, oh, underscore our rdr Rdwr.
Is reading right. Oh, underscore Wrt, I think, is right.
This, machine? Yeah, I don't. I don't normally run mate, what is on this thing?
Yeah.
So I don't let me see. Hang on one sec.
Oh!
Stan you were talking about. You know how newer computers sometimes are more of a hassle.
Right.
Get Linux running on them? The answer, that is, Bs.
If you buy decent equipment, it just works. This is a an epic server that I bought 2 weeks ago.
I'm in the the the last throws of configuring it.
It's it's got 32 cores, 64 threads, a fair amount of memory, and the entire work it took to get Linux running on this the first time was stick a thumb drive into it and hit control delete if if you look at hardware
Manufactured by this is a super micro motherboard.
Super, micro
Any of the Gigabyte Asus, any of the High End service boards.
Well, invariably today have good Linux support, in part because, unlike the stuff from Dell, they go out of their way to you.
Standard hardware. It'll be broadcom to controllers. It's stuff that if you look in Linux you'll have drivers for it.
Day one you don't have to download anything. So a lot of the stuff is is pretty easy when you start dealing with stuff like Dell, the the user desktop environment is full of companies that cut corners and they use cheap offball stuff and that yeah, that's where you have problems with the linux
Drivers, the machine. I'm replacing is a I bought it 10 years ago.
I think the the Bios is something like 2011 it's it's a 4 CPU.
48 core Octaon server that I bought on ebay for a $110 because it was depreciated down to 0 5 years after someone bought it.
And it in the last 10 plus years. It it hasn't crashed once there are times when I got lazy about rebooting it it was up for a couple of years because I was I was sloppy about updating the kernel like I just compile the stuff and it
Runs. I I build the kernel. If anyone cares one of the things I can show you is how to go about building a kernel from scratch and finding what drivers you need and building the kernel and keeping all the crap out part of security is building a kernel that doesn't
Have unused modules for everything under the sun. So that people can't try and load them and enforce known broken software to execute in places it shouldn't it's not that hard to build a kernel.
If I was. I was playing around before watching 30°C is pretty cool, I mean, the machine doesn't overrun.
I do? Made a few minor modifications to it.
Hmm.
I took the fans, the jerks who manufactured it put the fans on the inside with with greats in front of the fan that stop the airflow, so I cut the little gratings off and stuff and stuff fans on the outside pull in the air instead of pushing it for various reasons
Pulling only fluids is more efficient than pushing them.
So what I was doing earlier today is asking, here it is.
What happens if I run 64 processes that are doing nothing more?
Than a loop, just for semi semi curly, curly in in pearl, and you'll see here I've got a 100% user on all of these.
So this is just running it. It's it's sucking up every every CPU cycle of every thread everywhere looking at it this is getting a little bit warmer, but the fans aren't making any noise yet.
They're happy. The Nvme drives are perfectly cool. And one thing you notice about this epic server. The the steps on the cpus are tiny
You'll see these things going up in little tiny increments.
This link does not use any more power than it has to be
Which is kind of is kind of interesting. How that's one of the ways they they kept it more power efficient.
A neat thing about Amd. Chips is the CPU cycles per Watt, or much much slower than Intel
This is, yeah, this is all the processes I started that that was listing the hardware.
But you can see these things are all running it pretty much the maximum clock speed.
This will hit just about 70 Celsius, and if if anyone listened there's no the machines a foot for me, and I can barely hear the fans
But getting this thing running was an amazing level of no trouble, because the High end, especially server motherboards.
Are just they're built with really standard base, you know, basic chips that work.
And the drivers for them have been around on Linux since forever.
And again you can see here still. We're now the the lead time.
It's it's close to a minute. So now the the load average is is creeping up to 64 is all of these are running.
If I go here
That kills them all. What's interesting if you watch here, the temperature is already down
These are the actual temperature sensors on the CPU.
That's how long it takes to cool itself. It'll be another few minutes before it's down in the thirties again.
But it's amazing to me how quiet and low power this chip is for the amount of horsepower that it's got on
All right. This is a classic way to check something in bash.
I just said for the for one to the number of cores do a pearl minus E of just for semi semi, curly, curly, which don't even. It's not.
So while one it doesn't even have to check the number one, it just moves.
That is, that is just a loop statement, and when it's done I just kill that many processes, and I know it's stop them
This is a good way, if you want to benchmark.
How well your your runtime environment, especially if you're running virtual and a virtual machine, you can start to see how much time is being stolen from your virtual machine because you're not running all the time.
So tracking stolen time of things this th, this pearl minus e, with the for loop, allows you to check.
It's the lowest overhead statement you can make on the CPU, because it it turns into assembly.
That's just moving
But now you can see these things are back down into the forties from 70, so that that the cooler works wonderfully
Why did a lot of times people use leftover hardware if you use consumer grade stuff, you're gonna have a lot more trouble.
The capacitors aren't as high quality. The power supplies aren't as high quality.
If you look on ebay, or you look on amazon.com a lot of times, you can find a a server.
There's a place called servers@homeorservethehome.org where it's people who run servers you know.
Server grade, hardware, and it's not that much more expensive, and you get stuff that just stays up and running.
And it's a lot easier to configure for Linux
Basketball.
Anyway, that's all I had, unless someone's got any more questions, or I put you on
A question question with respect to your comments here in the last 2 min.
You you said. This is a way of of looking what's being stolen?
What processor times being
Yeah, if think about it, let's say I'm running a virtual machine.
And I've got 5 virtual machines running on the same set of cpus
The kernel that is executing my virtual machine uses time.
So, my CPU clock, we'll have time stolen from it.
If I examine the CPU cycles, if I multiply the CPU cycles by the number of seconds that I got, I will get less than the wall clock time, because some of my time I'm in a virtual environment my wall call time.
Isn't real. My wall clock time is virtual because the kernel that's actually running the hardware has to use some of that time in order to control the process.
That has my cop use the kernel running in? Did that make sense the way I set it?
Yeah, yeah, okay.
That's still fine. And the way to look at that is, you look at the number of clock cycles you get, and you look at the clock rate and the number of second, the the number of cycles times the rate the cycles per second
You can tell if you look at the clock, the number of clock ticks that you've got, you divide that by the clock tick rate you will get the number of seconds of wall clock that you think you are running.
If you compare that to an external source with a wall clock like Ntp.
You'll find that on a virtual machine you don't get all of the wall clock time because you're sharing the hardware with other things.
You cannot speak.
You're a virtual machine. You're not supposed to know that other machines are on the hardware.
Yeah, okay.
So your virtual machine in your virtual space inside of the hardware doesn't see all of the hardware all of the time.
Does does that depend on the virtual machine, whether it's a virtual box or or Katie
No, it's it, that is, that is a fact of life for any virtual environment.
If you're using lxc, let's say I've got 8 cores on a on a processor.
Correct.
And let's say I'm running for virtual machines, that each link they have 4 cpus.
You'll swap them
Well, I don't have 16 cpus, so what's gonna happen when my, when the CPU in my virtual system prize to do input output tool file, it can be blocked.
And the hardware can say your virtual machine is trying to access a file.
I'm the hardware I'm gonna get you the bytes out of that file when they describe gives them to me.
You are hereby stopped because you're blocked
Remember, what's your virtual machine? Your kernel is running inside of a process.
It isn't controlling the actual hardware. That's true for Virtual Box.
That's true for kvm, that's true, for vmware.
That's true, for every one of these
So there's overhead in the fact that you're running inside of a virtual machine, and the the real hardware has to do bookkeeping to ensure that your piece of hardware gets what it needs
Does that make sense
Yes, it does. Thank you.
Okay.
That's the stolen time. If people care about that, and it's it's interesting.
If if Gary wants to buy me a beer, one thing I could do with this this new toy that I've got is, I've got a copy.
I got Kvm running on it, we could go through the process of of setting up a virtual machine. And Kvm, I can show you what that looks like.
I'd like to see. That is Stan
Hmm.
Sounds. Kind of interesting.
Right, oh, right! It's it's 9 30, my time and my wife is a physician.
She's on the early shift tomorrow, and I've got to get her dinner before and cause we gotta get up at about 50'clock in the morning.
So apologies. I just don't have more time for questions
Yeah. Sorry about that. Usually. I'll say here, one thing, yes.
Alright! Well, good, for not good, but we understand so alright
People care about
If people care about what's going on on your machine, there's a thing called sensors
It'll probably be called Sis app selling sensors or something.
If you look for Ln. And sensors, this is the thing we'll allow you to check.
Tem, mostly temperature. But if you've got equipment on your machine that's managing voltage regulation, anything else, any of those sensors that are visible to Linux, are are gonna be available through here if you like.
Playing with raspberry. Pi's, and you want to play with all those extra pins on the edge.
This is where you'll get information from those
So this is, this is actually a pretty good thing, especially if you're stress testing machine.
You can run a watch of sensors, and if you're gonna cook the cpus, you'll know about it before you start smelling a zoom ozone, which is a nice thing to do with a $1,000 chip
That's great. Thank you very much for the presentation
Oh, glad! It helped, anyway. My apologies. I've got to get going before my wife Springs.
She's a doctor. She knows how
Alright, no problem
Good luck to you! Thanks again
Thanks, Steve.
Thank you, Steve. Thank you.
Somewhere. He
For the rest of you, even though Stephen had to run.
We we do have that discuss mailing list, and I'm sure Stephen will be happy to answer, or other people might engage.
If you have any comments or questions, post them and discuss and we'll continue this discussion.
Actually, even though our speaker has left. If anyone carried a bandy about a couple of comments, questions about what we just heard, I imagine a few people would be willing to hang around here a few minutes, also, if anybody wants to wander off into some other general topic, we could also do that at this
Point, so
Anybody talk.
Talk, talk, talk.
Well, like I was saying I got. I got the my laptop.
It's actually, I think, ban.
And I. I needed something to, you know. Scan onsite for the with Gbm.
So I did install actually a
I think I think it is a bond to, and then I I ran some like a virtual machine, so I can run Kelly inside the ubuntu that actually seemed to work.
Okay. It's fine. And it didn't really take me that long to install it.
I did. I think I did run into a complication, but it wasn't that difficult to go around.
It. There's a lot of information online
If you would. Good with your searches online, then you can find out a way to resolve your hardware problems
You know. And now, of course, everything's running, and it's been running for a half a year or so
I always suggest asking Phil to help you make a a search engine queries, because I'm no good at it.
Well, now you can do is chat, gpt, you know, and as long as it's a old hold information it will find it
True.
Anyway. Well, I'm I'm gonna I have also things to do.
Unfortunately. So I will send you guys. I'll send Gary the
The thing that I'm thinking about the Bci. The Pci is, you know, payment card industry.
If you if you're at all using credit cards in a business, you have to, you should be compliant with Bci standards.
No, it probably depends on what kind of business you are whether you are a big stickler with it. Or if you are just basically doing what the minimum things
But the the new version was released last year, and it actually has to be implemented by 2,025.
There are actually a number of things that I have changed, including in the network security angles.
So that's kind of what we are doing to talk about.
I unfortunately, it's not going to be a technical thing.
It's just, you know, the more the management angle of it I'll I'll think about if there are any technical at.
And now items. But anyway.
Sometimes that's interesting. Sometimes that's interesting to management stuff.
And so thank you for your comments about the loading Linux.
That's interesting. Also.
Yeah, I mean, there is a lot of information out there, even if you're having problems, you should be able to resolve it.
And people do help you.
You. You just have to learn how to connect with others like, you know.
Gbm. Has a it's an open. It has the open source.
It also has the paper. So a lot of times they will say, well, we don't handle that.
But you can go. Ask Kelly. You can go on the call Forums, and they will help you.
So you know, there is ways to get information just can't give up, that's all.
Anyways, alright, very good. Thanks thanks for the it was good, interesting!
The estrays
Thank you, Tony.
Alright! We'll see you
Thanks. Shannon.
Well.
Any other questions about the presentation or comments, or anybody want to lead off on talking about it.
Yeah, I have a think that was mentioned, and I was looking around.
And I think I'm getting a used to. You know.
I end machine something like a p. 52. But does anybody have any tips for you know what type of security check should be done if you get something like that, you know, to the extent such a thing as possible?
And I'm sure, like, you know, if some you know 3 letter agency were to put something on it.
Yeah, maybe you know, that would be difficult to detect, but short of that, what are the kind of commonsensical things that can be done?
If you were looking to get a use imped or something along those lines right
As far as software to run. I always recommend well for myself, always make sure to run, fail to ban right away, and then root.
Kit Hunter is a good tool, and there's another one I can't think of off hand.
But here they're pretty commonly mentioned out there
If you're running any services on it, learn app armor
Yes.
That's the best way to secure the services that you're running.
It restricts the execution, and the directories at a server can access
Cool. Hmm
I recommend we and and
Sam, there's a comment that Phil B.
Just put into into the chat. So he responded to your question.
But it's it's in written farm. So look in chat, and he'll see
Sure, if you have any, you know anything else going on, just post and discuss, and you get a bunch of people at the same time.
Alright!
With a with a lot of different levels of experience, and you might hit the jackpot, and there'll be discussion going on for days, and you'll learn more than you even hope to learn sometime
I suggest ending the formal recording any or their more discussion here
Sounds good to me, Gary, you wanna do the book end
The way I need to apologize. I've I just more carefully read Phil's comment, and it really wasn't in direct response to your question, Sam.
It was about an articles about S. Trace.
If Youulie Evans wrote so oh, sorry about that!
And answer to Lee's comment, Yeah, I guess we can shut off the recording.
23. General Meeting of Slug. The main topic tonight was S.
Trace, particularly desk phrase peeking at what zoom is actually doing.
And our basic tutorial on the main speaker was Stephen Lombard.
Are.