I'd posted about recently being hacked in a thread on texting-based customer support in the Moan Zone, and member Kpatz had the great suggestion to start a dedicated thread about member's experiences with being hacked, and mitigation / recovery strategies. I'm a product & web designer by trade, and dabble a bit with programming for my wife's company. I'm interested in technology and enjoy working with it / learning about it, but am not an expert in any way with networking or the inner-workings of computers. But I know a bit, and will start this thread with a discussion of what happened to me and how I'm trying to fix it. I'll follow-up when I have time with some photos I've taken over the course of the hack showing some of the footprints left by the attacker. I know there are a Microsoft employee or two who are frequent contributors in this forum, so hopefully they'll chime in with their expertise on suggestions for keeping our computers safe.

I have a remote Ubuntu server hosted on Linode where I keep my portfolio and web programming projects I occasionally do for my wife's company. I've been out of full-time work for the last five years caring for elderly parents, and when Covid hit I didn't access the server much if at all. It was a Ubuntu 16.04 server that had few if any updates applied during this time.

In February, I started working on a new project for my wife's company, so began updating the server and posting a PHP / MySQL / JavaScript-based project that my wife's employees could access. I immediately began noticing strange behavior on the server: text typed in my shell was very laggy; connections kept breaking (especially when modifying configuration files); etc. Rather than fight with the server, I decided it was time to start fresh and I wiped the old server and started with new virtual disks and a new Ubuntu 20.04 server.

A day or two later (March 9) I logged onto the server just before going to bed to FTP some changes to the project when I made a fateful mistake: I clicked through an SSH (secure shell) prompt that my server's keys had changed: a classic man-in-the-middle attack. I immediately knew I'd made a mistake, but wasn't too worried about it because I was using a Yubikey for two-factor authentication on the server. I mistakenly thought that would protect me.

I knew something was seriously wrong the next morning when I logged back on and wasn't prompted to touch my Yubikey to authenticate. Investigation revealed that the attacker had deleted all the configuration files associated with the Yubikey.

I spent the next week trying to reclaim my server, while blithely ignoring that the attacker might also have infiltrated our home network. In my next post, I'll talk about what I observed when I realized nefarious activities were also afoot at home ...

At the time we were hacked, our home network consisted of an Arris cable modem; pfSense router; a Windows 2016 server for computer backups; three windows PCs; two mac laptops; a Ubuntu server; a Ubiquiti Unifi wireless access point; a handful of Rokus; and several Extollo powerline ethernet adapters. I hadn't previously thought of the Rokus or powerline adapters as computers, but they are: they have embedded Linux at their core, and are thus equally (probably more) vulnerable to being hacked than the dedicated computers.

I first noticed something was wrong on my Mac laptop: when looking at my Applications folder in Finder, the only application it showed was a network vulnerability scanner called Burp Suite. When looking at the Applications folder through a BASH shell, all my expected applications were there.

I then noticed my Recycle Bin wouldn't empty, even through the shell, and there were a bunch of weird ACLs on everything below my user directory: I was effectively locked out of accessing certain directories or deleting certain files.

A couple of weeks before I began having problems on the server, I'd gotten a brand new tricked-out Dell XPS Windows 11 PC, and had just finished downloading / installing / configuring my DAW and tons of VSTs. Literally two weeks of work getting everything as I wanted. I was briefly in heaven - the machine was a dream, and much snappier than my previous Dell which I'd been quite happy with.

The first thing I noticed wrong on my new PC was that the attacker had renamed my "Application Installers" directory to the more Linux-friendly "Application_installers", and he'd deleted everything that might be useful in my recovering from the attack (operating system images; gParted for disk partitioning / repair; etc.).

And then I began noticing strange behavior on websites I frequently go to. As a designer, my attention to detail is pretty good. After restoring my new PC to factory defaults and beginning to reinstall everything, I went to the Enpass (password vault) website to download the app. The site looked subtly different to me: the button of the active tab usually has a white text on a reddish-orange background. Everything else on the site looked identical, but the text on the active tab was gray. I noted it as an oddity but downloaded the app.

To make my life more convenient, I synced my passwords to an encrypted Dropbox folder so all computers would stay up-to-date. After reinstalling, sync suddenly stopped working.

Then I noticed the next spoofed website. My wife and I recently began a bathroom remodel, so I needed to transfer funds to my bank to pay the contractor. When I went to the Vanguard website, I noticed that the password field was highlighted in red before I'd entered my password (the red highlighting is a typical syntax for indicating a mis-typed password). At this point I thought I should move to a safe neutral network with a refreshed mac before going any further. I went to my Mom's house and changed all sensitive passwords.

When I woke up the next morning, the attacker had changed the master vault password. I went back to my Mom's house the next day (Sunday) and changed passwords once again. When I tried to open the vault later at home, I was locked out again.

By now I was panicking, worrying that the attacker would be able to wipe out my retirement and investment accounts. My main financial institution is on the East Coast and I'm on the West Coast, so Monday morning I set an alarm for 4:45 so I could place a call to lock my accounts.
Shortly after that while telling my wife I'd locked the accounts, my phone's camera turned itself on. More on the saga and beginning steps to mitigate later today. Pictures at 11.

After freezing my accounts, I began trying to figure out how the hacker kept getting back in after I'd reinstalled all operating systems and the router. I reached out to friends on Facebook asking if anyone had recommendations for a security professional. No real leads came from that, but one of my friends mentioned he'd helped his father in law recover from a similar experience. He suggested replacing all my networking equipment which sounded reasonable. I got rid of the powerline adapters and got a screaming-fast new Nighthawk cable modem with built-in wifi. My wife and I took our macs and iPhones to the local genius bar and had them wiped / refreshed. I bought a new iPhone, but held on to the old one until I could transfer all my two factor authenticator apps and make sure that was all working as expected (more on that later: if you aren't using two factor authentication already or don't know what it is, LEARN ABOUT IT - I think it was instrumental in keeping the hacker partially neutralized, at least with respect to compromising my sensitive accounts in a meaningful way).

The hacker was back inside our network within a day. I called Xfinity and the call was elevated to their fraud division. When looking at the IP address for my cable modem, Comcast was showing a different IP address than what I was seeing in pfSense.

In looking at netstat network data, I was seeing some suspicious connections to my computer. One was to a DNS site, which when researched on Internet WhoIs revealed it to be a known malicious site with over 5,000 complaints logged.

Since having our macs refreshed, they both seem (so far) to be working as expected. But the PCs continue to be a problem.

My first clue at how the attacker was accomplishing things was when I attempted to make a bootable Windows 10 USB stick on a 16 GB drive. This should have been more than sufficient space to create a bootable USB, but Windows was reporting insufficient space.

I began researching forensics tools to start trying to figure out how the hacker kept getting back in. I read good things about R-Studio and it was reasonably priced, so I figured I'd start there. It scans drives for deleted files and hidden partitions. I recovered all the files I could and pointed my virus scanner at them. It found seven severe malware infections: I'll post pictures later of the seven, but the name I remember from the seven is called Dorkbot, which self-propagates over networks and infects attached USB devices. It also hides itself in so-called slack space (all files in Windows have a specific block size on a given partition, so if a file doesn't use all the reserved space, this slack space can be used by attackers to hide useful files and tools).

Now having a bit of an idea about how the attacker kept coming back, I began trying to wipe my drives. When you delete files on a computer, they actually aren't deleted: the OS just changes a flag that indicates the space should no longer be reserved, and can be over-written if space becomes scarce. Similarly, formatting a drive doesn't really get rid of the files. I looked into the best way to wipe the drives to remove all traces of the hacker, and found an app that could be booted from a USB stick to wipe the drive. Whenever I tried to write the files to USB, the app I was using to write the files kept reporting that there were no bootable files.

I next tried to use Window's built-in DISKPART to accomplish the same thing, but that had been modified by the attacker as well.

One other tool I used to try to gain insight into what was happening on my system was Microsoft's Process Explorer. When I launched it on an infected machine, it showed many strange and altered command line flags for Winiit, Firefox, and others. I'll post some pictures later.

And that, ultimately, is as far as I've gotten a little over a month into this saga. For now the Macs seem to be operating OK, and I've got to figure out some way to get my drives wiped so I can try to get traction with the Windows machines. Oh, one other tell-tale sign that the attacker left on all the infected machines was extra unallocated space after partitions: that is where many files are being hidden I think and how he keeps reinfecting.

Do you have a DVD or BluRay burner (a USB one)? I'd download fresh Windows 10/11 ISOs from Microsoft and burn them to discs on a KNOWN CLEAN system. Verify the SHA/MD5 hashes before and after burning. Then use those to reinstall Windows. Do the same for any Linux ISOs.

USB drives have the distinct disadvantage from a security perspective of being writable, which means any malware can alter or infect the drive, and spread it to other systems.

A secure wipe of your affected hard drives should make them reusable, assuming there was no vulnerability that allowed them to alter the firmware on the drive. Maybe get new drives, wipe the old ones, put them in USB enclosures and use them as data backup drives rather than OS drives to be safe. Make sure all partitions are gone/overwritten including the EFI/UEFI partition and the MBR/GPT is wiped as well. I'd wipe them from a non windows OS like Linux. Even the dd command, writing /dev/zero to the entire drive, should do the trick, if the drive doesn't support secure wipe. (Note, if it a SSD, look for a suitable wipe tool, as they may have special commands to completely wipe in an efficient way... check out blkdiscard in Linux).

Maybe in the future, set up an isolated VM to use to access your remote servers in case they get compromised again. If you snapshot the VM and restrict its access to the rest of your network, if a hacker gets in, they will only affect the VM, and reverting the snapshot will restore it to a clean state.

You weren't clear as to the ownership of that copy of Burp Suite.

Is it yours or was it installed by the actor?

Burp Suite is much more than a vulnerability scanner.

At it's core it's a web capturing proxy. It's likely this is how you're being served up these fake sites.

He's running it in the background via command line.

You type google's URL ( or your bank etc) into your browser's address bar and he's likely got Burp configured to resolve to one of his "creations" on Amazon AWS or Linode etc..

If you have any questions RE Burp Suite, it's a tool that I've logged thousands of hours on.

You should also be using something more low-level to capture ALL protocols, like WireShark or similar.

By the hacker.

I've been trying to use Wireshark, but to decrypt the traffic, he's blocking my ability to do so. I tried yesterday on Ubuntu using the secure-key.log option, but he (or she) was redirecting my Firefox download: the instructions presented to me had me add an apt repository, which apt-update than complained about because the repository wasn't properly signed. He's also redirecting all my attempts to download any antivirus other than clamav, which seems to have installed correctly via apt install.

At my wist end at this point - I know which virusus / malware have been installed, but every tool built into Mac OS, Ubuntu, and Windows for zeroing out my drives and USB devices he's corrupted.

The last couple of days I've been trying to find someone in the Bay Area to help since nothing I'm doing is moving me forward in a meaningful way. Nobody so far is willing to take on the project, as they say it sounds like the attacker is in very deep and many are just re-normalizing their businesses after Covid and don't have sufficient staffing to take it on. I've had three different companies tell me my best bet at this point is to just start over, with completely new equipment, to be sure the attacker is really gone. After beating my head against a wall for over a month, I'm starting to see the logic in that.

I've ordered all new network equipment, but am waiting on installing that until I can get at least one machine stabilized. Since that doesn't seem to be happening, I just ordered a new Mac laptop and the new Dell I got is still under warranty, so I'm shipping that back for servicing. Hopefully that will finally turn the tide.

I don't know much about Burp Suite; I just saw it in my Applications directory and looked it up once I saw it there. That probably was initially how he was sending me fake sites. Maybe he was using Burp Suite to do it, but at one point I brought up the same site in two different browsers on the same computer, and the document tree was completely different when viewed in Developer Tools across the two browsers. One he'd somehow injected additional content (JavaScript and other content - don't remember now but think I snapped some pictures on my phone).

It will cost you way less time and agravation to get completely new hardware.

Especially do NOT trust any drives - I think I don't have to explain why.

All that injected content can be automated using Burp Suite.

Burp allows you also manually view / stop / edit request and responses on the fly or automate processes.

And since it uses a self-signed cert, you can view your own requests & responses without the SSL encryption.

Which also (sadly) means that it's very possible he was able to see passwords / creds in cleartext.

To be clear, I'm only able to speculate what he was doing exactly with Burp Suite but he sure didn't drop it on you while trying to earn a merit badge.

I really feel for you man - this has got to suck so bad. I know If it was me, I wouldn't be able to sleep AT ALL until I had the upper hand.

Good luck!