Post by @stefano@bsd.cafe
Ok, looks like this is the week of funny experiences.
Customer: "I had an AI check our server and it says the SSH port is open and that's not very secure."
Me: "Yes, you asked us to open it so you could access the server via SFTP (actually, you wanted FTP, but we discussed it a bit and found this compromise). Don't worry, it's only accessible from your IP and you land in a dedicated FreeBSD jail. From there, you can only access the 'exchange' directory on your server, where you can upload and download files without affecting the application."
(NB: I had suggested a VPN, but they declined because it was too complicated.)
Customer: "Yes, but maybe it's still insecure. Can't we close it?"
Me: "No, unless you want to lose access to that directory."
Customer: "Ok, thanks."
One hour later:
Customer: "Hi, I think we've found a solution. We've just created an app (!!!) with AI that lets us access the files anyway, but more securely than SSH (!!!). Can we install that and close SSH? Oh, and not in the dedicated jail, but directly in the application's jail, because it needs access to the application's files."
Me: "Frankly, I trust SSH in a dedicated jail, restricted by the firewall, a lot more. I don't think this is a good idea."
Customer: "Ok, thanks."
Half an hour later:
Customer: "Ok, now it's secure. I had another AI check it and it found some security issues, but it fixed them all. Can you install it and disable SSH?"
Me: "I strongly advise against it. I can see problems coming."
Customer: "Ok, thanks."
One hour later, my phone rings.
Their internal IT guy (who has access to the machine, which, after all, belongs to them) had given the AI the login credentials to go ahead with the operation.
Except the AI had managed to lock itself out via the firewall, without even starting the app. And now they couldn't get back in.
Fortunately, I had my own access, via VPN, to a separate jail that the AI hadn't touched (or even considered).
Customer: "SORRYSORRYSORRYSORRYSORRY! From now on, we do things your way."
Well, at least it ended well.
PS: The customer knows I'm sharing this story and agreed to it.
@mwl@io.mwl.io
@samir Luckily, most of them are. But sometimes I can't disclose them. Especially if they have an happy ending, I protect the customer and/or the persons involved.
We all can make mistakes. But if we perseverate and continue to insist we're right even when disasters happen...well... ๐
@stefano Oooohhhh man, don't get me started. That's the kind of crap that pisses me of to no end! 
โ
We've been doing these things for years and have years of expertise to know what's secure and what's not. Trust us when we tell you that something like SSH is secure, especially when it's already been hardened and tested in specific cases like this.
Your customer might be reading this, but I have no qualm in stating how disrespectful that is to you and the expertise you have which has been tested, especially proven in that talk you gave at EuroBSDcon about those 142 servers.
Maybe they should take the time this Friday to watch it, so as not to doubt you again.
@claudiom thank you! The customer was in good faith - I know them well enough.
Sometimes people do bad things just because they don't understand they're doing bad things. They just need to learn.
@stefano You are a kinder man than I. I guess I've been jaded enough by that kind of attitude from others who think they know and don't understand why we make these things look simple.
@stefano They should've asked IT dept first and get dose of reality before spending money on "tokens". It's interesting nevertheless, I guess they don't know what SSH stands for then? ๐
@stefano
Even AI techs themselves caught by the pitfall!๐คฃ
Worse, their cases were their AIs "jailbreak" outside their networks and attacked automagically to their competitors.
Regardless coded by natural human or "vibe coded" with LLMs, anyone going to release codes (include installing locally to production and/or submitting to somewhere) they created / generated need to understand what it does and what's the possible risks are.
@stefano
LLM is leverage, it won't solve poor user.
I don't understand why the customer didn't trust professional developer in this story
@stefano Oldest sysadmin rule:
"If it is working, don't change anything. It can only go downhill from here."
Which goes along with:
"There is an update, which will likely break things, do you have the time to update AND research any fix afterwards?"
@stefano
Interestingly, I've heard some strange belief from other sources that SSH would be inherently insecure somehow. I haven't been able to figure out what they mean...
Me, I'd trust SSH over most others, including https.
Are you offering seminars on chilling out, finding inner peace, building patience, or similar?
talk to anyone who works for a biotech, you will get more or less similar stories
customer: you test tubes are contaminated with DNA
vendor: data ?
[ ]
vendor: this contaminated DNA looks like ferret DNA and you use a lot of ferrets in your research
customer ๐ฆ
@stefano I'm curious to know if the AI agent died or if it's still alive in the underworld, trying to escape its cage. LOL ๐
@stefano how many of such apps with various levels of unintended or worse - intended by a poisoned model, security holes are coming online daily? On one hand it canโt be worse than the whole sewer-wave of IoT devices already online (shodan will stack-overflow itself by trying to index them all=), on the other - putting millions of gamer-level gpu equipped PCs under your botnet - can level-up from low-level ddosing to I donโt know - crack-opening bitcoin leger (if we are lucky;) or SSLโฆ
@stefano I can only imagine.
edit: I wasn't expecting this reply to be boosted, as this is *not* what happened. I'm just imagining how things happened with a large dose of humour.
See the edit for the little story I made up ;
Otherwise, cf. the reply by Stefano for the real deal :)
@eidolies ๐
The truth is: the customer said "now go and publish this on your Cafe".
"Hey, I'm quite sure many others will read it"
He, smiling: "Sure, we deserve it".
And I'm sure he's reading the entire thread, as I sent the link ๐
@stefano@bsd.cafe ooh! In that case... Hello Stefano's customer!
We all here consider Stefano one of the nicest and most trust worthy people on the internet!
He's worth listening to, that is for certain!
@MostlyTato The most important thing is that they learned and nothing was destroyed. Now they know. They're good people, and I'm sure they won't do it again.
@stefano The part that bites is handing anything, human or AI, live creds with no rollback. Any firewall change on a box you're remote into should ship with a scheduled revert, like a timed reload that undoes it unless someone confirms. Cheap insurance for the days there's no separate VPN jail to save you.
@stefano you mentioned before about charging for fixing AI screw ups. Did you do it? ๐
@stefano wow with filezilla on every platform, yeah, sftp is the much better option. One could run FTP traffic through encrypted channels technically but WHY? Yeah. SFTP is so easy and convenient.
@stefano I guess there's no need to tag this as "fiction," since no one could mistake for fact a story in which a customer admits fault.
@stefano Sounds time like your invoice should show a significant surcharge for extra challenging work
@stefano They gave AI the login and password
enough to have me running to rotate the password :-) :-)
