Day 4: Busting Root on the Apple

Flir wandered back to the quad at lunch , eager to count his password stash and see what Set-UID shells he d gained since yesterday . He logged in to mac3 again, now using another name and password picked up by the sniffer.

 # ifconfig eth0:0 10.0.50.57 $ ssh -b 10.0.50.57 [e-mail] 

He first got a list of his Set-UID shells:

 [mac3:~] griffy% ls -l ~mrash/Public/Drop\ Box/.shells  grep zsh -rwsr-xr-x  1 arthur human 828780 Apr  5 10:32 zsh-arthur -rwsr-xr-x  1 ford   human  828780 Apr  4 22:55 zsh-ford -rwsr-xr-x  1 steve  staff  828780 Apr  5 00:01 zsh-steve -rwsr-xr-x  1 wstearns   human  828780 Apr  5 07:02 zsh-wstearns -rwsr-xr-x  1 zaphod human  828780 Apr  4 16:42 zsh-zaphod 

Flir s eyes flew to the zsh-steve shell, fixating on the staff group. The staff group on OS X indicated one of the administrators on the machine and usually got a good deal more privilege.

Flir ran the shell and felt a mixture of fear and power grow over him:

 [mac3:~] griffy% ~mrash/Public/Drop\ Box/.shells/zsh-steve mac3% 

He instantly thought to run the nidump program, which he hadn t been able to run earlier because of the permissions. He ran it, hoping to get password hashes for the rest of the users on the system:

 mac3% nidump passwd . > ~mrash/Public/Drop\ Box/.shells/hash mac3% chmod 755 ~mrash/Public/Drop\ Box/.shells/hash 

Flir read the file to confirm that it was getting hashes:

 mac3% less ~mrash/Public/Drop\ Box/.shells/hash nobody:*:-2:-2::0:0:Unprivileged User:/dev/null:/dev/null root:*:0:0::0:0:System Administrator:/var/root:/bin/tcsh dna:0NX4GcExbdraU:501:20::0:0:Doug N Adams:/Users/dna:/bin/bash aadam:a4IemqRpsQKL2:502:20::0:0:Andrew Adams:/Users/aadam:/bin/tcsh andyb:3p/6EIfCfP4z9:503:20::0:0:Andy Brendan:/Users/andyb:/bin/tcsh 

The names and passwords streamed on and on. He checked the line count:

 mac3% wc -l /etc/passwd    40823 /etc/passwd 

Flir couldn t believe it, though he d know this was the consequence of simply running nidump on the system. He had password hashes for over 40,000 accounts. Given how badly people picked their passwords, 50% to 75% of them could be cracked, given sufficient time and computing power. That was two to three times as many accounts as what he was going to get out of the web application man-in-the-middle attack. He might not even have to keep intercepting logins if he could just figure out how to crack those passwords in a reasonable amount of time.

There was more than that, though. If he could crack this admin s password, he could get root. As root, he could alter the entire environment for anyone who logged in. He could install keystroke loggers, read e-mail, or even just kick everyone off the system. But that was getting ahead of himself. He hadn t cracked steve s password yet, and he might not ever be able to do it, if it was well- chosen enough. For now, he d focus on cracking all the passwords, paying special attention to this one, but not relying on it completely.

He exited the steve shell

 mac3% exit [mac3:~] griffy% 

and began to think about how he might crack 40,000 passwords. He considered the Physics department s computational cluster, but it was constantly maxed out. Physics wasn t exactly rolling in grants after losing a professor to criminal fraud charges last year. Flir didn t like thinking about that though ” he wanted to put the famed Popcorn Incident behind him. Besides, using a shared cluster on campus wouldn t be too stealthy, especially if he had to use his own account there. He d need to think of other options.

He went back and looked at his collection of Set-UID shells. There were 23 now, but one stood out from the rest.

 -rwsr-xr-x  1 wstearns   staff  828780 Apr  5 07:02 zsh-wstearns 

All of the other shells had creation times that mapped times when students were usually logged into the system, but this one had a creation time of 7:02am. No self-respecting student would be working on the computer at this time unless he was still awake from the night before. No, this was almost certainly a professor.

The name wstearns stood out in Flir s mind, so he did a campus directory search and found that the account belonged to a visiting professor in computing, William Stearns, from Virginia Tech. Flir checked Professor Stearn s process list and found that he was ssh-ing back to a machine called gateway.cluster.vatech.edu:

 wstearns   2569   0.0  0.1     1792    608  p4  S+    6:44AM   0:00.22   501  566   0  31  0 -      ssh [e-mail] 

Right, Flir thought, Virginia Tech just built that huge cluster of Apple G5 towers . They built themselves a supercomputer! Wanting to learn more about what his accounts could do, and Flir was already thinking of these shells as his accounts, he ran a google search on Virginia Tech supercomputer and found a link to the site for the Terascale Cluster at http://computing.vt.edu/research_computing/terascale/.

He clicked on the Slide Presentation link and started to read details on the cluster. It was the 3rd- fastest publicly known supercomputer in the world, behind the Earth Simulator Center and Los Alamos. It had 1,100 computers, or nodes, each of which had two 2Ghz G5 processors, 4GB of RAM, and a 160GB serial-ATA hard drive. Each processor had its own independent memory bus, allowing the processors to work more independently than comparable multi-processor PC s. The machines communicated by 20-gigabit network cards. They ran Mac OS X and supported MPI, the Message-Passing Interface library that the scientific computing community had standardized on. MPI made cluster computing far easier, allowing each processor to communicate with its siblings on other machines without having to use hardware-specific mechanisms.

All of those specs aside, Flir was in shock . He was about to gain access to the third-fastest publicly-known supercomputer in the world, because of a simple permissions problem on his school s Xserve and the fact that this professor was running ssh from a shared server.

He realized that the best way to stealthily trojan Professor Stearns ssh was to replace his ssh program with one that logged keystrokes, but only Stearn s ssh. Flir checked the ssh version string, primarily to learn which SSH variant mac3 used:

 [mac3:~] griffy% ssh -V OpenSSH_3.4p1+CAN-2003-0693, SSH protocols 1.5/2.0, OpenSSL 0x0090609f 

Flir downloaded source code for OpenSSH, read through it well enough to find the point where ssh encrypted the data it was to send out. He inserted three lines of C at the beginning of the routine, so it would append the data to a file just before beginning the work of encrypting it. Of course, there were more elegant ways to log keystrokes than modifying the ssh code, mostly involving modifying the running kernel. Flir wasn t comfortable with these techniques because they were far more complex and intrusive . This increased both the risk that something would go wrong that could disrupt the entire machine, and the somewhat related risk that Flir s actions would be noticed. Flir didn t have root access, so the kernel options weren t open to him, but he wouldn t have taken them if they were. Flir recompiled his ssh client and now needed to ensure that the professor would run his client instead of the primary system one. He copied the shell into the Drop Box directory he d been using for all this time:

 [mac3:~] griffy% cp ssh ~mrash/Public/Drop\ Box/.shells/ 

He then ran his wstearns shell, to assume the identity of the professor.

 [mac3:~] griffy% ~mrash/Public/Drop\ Box/.shells/zsh-wstearns mac% 

He copied the ssh binary into the professor s ~/bin directory, /Users/wstearns/bin:

 mac% cp ~mrash/Public/Drop\ Box/.shells/ssh ~/bin/ 

Finally, he needed to modify the professor s PATH to look for binaries in the ~/bin directory first. This would ensure that the professor would run the trojaned ssh binary, without requiring Flir to modify the systems more globally.

He checked his passwd dump and confirmed that Stearns used bash and then added his PATH modification to the end of the .bashrc file:

 mac% echo "export PATH=$HOME/bin:$PATH" >> ~/.bashrc 

Now he just needed to wait for the professor to disconnect from the cluster and log in again. Actually, the professor would need to start a new shell first, probably by logging in again. Flir would either need to wait for Stearns to disconnect from the mac3 or force matters himself. He decided to knock down Stearns login. It was 2pm “ the professor would probably log right back in.

He ran a ps command to get a listing of the professor s processes.

 mac% ps auxl  grep wstearns root        565   0.0  0.0    14048    196  p1  Ss    7:03AM   0:00.65     0   421   0  31  0 -      login -pf wstearns wstearns    566   0.0  0.1     1828    460  p1  S     7:03AM   0:00.79   501   565   0  31  0 -      -bash (bash) wstearns   2569   0.0  0.1     1792    608  p4  S+    6:44AM   0:00.22   501   566   0  31  0 -      ssh [e-mail] 

The shell from which all his other processes had been started was process ID 566. Since Flir was running as user wstearns, he could send terminate signals to the professor s processes. He shut down the professor s primary shell, disconnecting him:

 mac% kill -9 566 

Professor Stearns did log in directly afterwards, reconnecting to the cluster. Flir collected the password that Stearns used, mason30firewall, removed his trojaned ssh binary from the professor s home directory, and exited his Stearns shell:

 mac3% exit [mac3:~] griffy% logout 

He then dropped the aliased IP again and disconnected from the rogue laptop:

 # ifconfig eth0:0 down 

Flir needed to take a break now and think about how to get the cluster. He could login to the cluster later, after Professor Stearns and most of the other scientists had stopped working for the day. For now, he would need to research cluster-based password cracking.

Researching the Password Crack

Flir ran a google search on distributed password cracking and come up with two papers and two good tools. The first paper detailed Teracrack, the San Diego Supercomputer Center s (SDSC) 1999 experiment in password cracking. The SDSC researchers used their cluster, Blue Horizon, to compute and store each of the 4096 crypt() ed versions of each word in a 51 million password dictionary.

Once they stored the hashes in a 1.1 terabyte database, they could check any crypt- hashed password against the table. If the crack program could discover the password, that password s hash would be in the table, pointing to the real password. Most users passwords would fit into their dictionary, so long as the organization did not require particularly strong passwords. The researchers had created their dictionary by combining the UNIX dictionary with the Crack program s dictionary, yielding 1.2 million passwords, and then using Crack s routines to apply manipulations and permutations to generate about 50 times as many passwords.

The scary thing was that the San Diego cluster could generate the entire table in 80 minutes. Terascale could probably do it in 7 minutes, given that its G5 processors were more modern, about 5 times as fast and almost twice as numerous . And while the 1.1 terabyte table had required a good portion of Blue Horizon s 5.1 terabyte RAID array in 1999, it wouldn t even consume 1% of VA Tech s 176 terabyte array.

When Flir realized that he could do that in 7 minutes, he also thought about what he could do for a few important passwords: a partial brute-force. He could take the salt for a given password and compute the hash of every possible password. Unless he restricted the composition, though, this would be fairly infeasible, still. If he only looked at passwords that used only lower case characters and numbers , though, he only had 2.9 trillion (36 + 362 + + 368) possibilities.

He began to read papers on distributing crack processes across nodes in a cluster. Based on an estimate of 500,000 hashes per second per processor, or 1.1 billion hashes per second for the cluster, Flir thought he could crack a password that used this reduced character set in 44 minutes. This would require no disk space and would probably do most passwords in about 22 minutes. He couldn t do that for every password, though, since it would take around 122 days of full-out computation at worst ^[iii] .

So once he logged onto the cluster, Flir figured he could crack about half of the 40,000 passwords on campus just by spending 7 minutes computing a table and then looking each password s hash up in the table. Those table lookups would take time, but Flir could optimize that by storing the 4096 tables that were being computed separately. These tables would only be 268 megabytes each. He d only have to search each of these tables for 10 passwords, on average, so it wasn t worth sorting the tables.

Flir wandered back to his room to write the programs. He d use them later in the night, around 6 pm , once Professor Stearns was logged off and most of the computation started by the professors back at Virginia Tech had finished.

Time to Crack Some Passwords

At 6 pm , Flir walked to the campus restaurant to eat dinner and to run his programs. He d normally eat at one of the dining halls, but none of them were very close to the computing building. The campus restaurant was even closer to the computing building than the quad, so it was conveniently located, even if the food was fairly routine and uninventive burger-pub fare.

He logged in to the mac3 machine with another one of his stolen accounts, and switched over to his wstearns context by running the wstearns shell:

 [mac3:~] ajr % ~mrash/Public/Drop\ Box/.shells/zsh-wstearns mac% 

He next ssh ed into the VA Tech cluster using wstearns password:

 mac% ssh [e-mail] [e-mail]'s password: 

He typed mason30firewall and was granted a bash shell on the gateway machine from which a user could start a cluster program. Initiating a sequence of sshs, he copied his program from his remote laptop to the rogue machine, from the rogue machine to mac3 and from mac3 to the cluster.

 $ cat program  ssh kent@rogue "ssh [e-mail] \"ssh gateway.cluster.vatech.edu 'cat >program' \" " 

He started the run and thought back over his design.

Instead of simply writing the 4096 268-megabyte tables to disk, though, Flir had made a crucial optimization. Each 2-processor node would keep its two 268 MB resultant tables for a given salt in memory, checking the 8-12 hashed Pacific Tech passwords for each salt against the corresponding table. It would then discard those two tables and do the other pair. Since each node had 4 gigabytes of memory, this only consumed about an eighth of a node s RAM and hopefully wouldn t trip resource alarms.

Flir had made one other optimization. As the 51 million hashes in each table were computed, their index in the list was added to one of 4,096 linked lists corresponding to the first two characters in the hash. This indexing reduced the number of string comparisons per password to 12,451. Finding which linked list corresponded to a pair of characters was similarly easy, since those characters were equivalent to a 12-bit number and that equivalency could be computed easily. The resulting code was fast.

Instead of a 7-minute run, the program took 20 minutes. Instead of producing a table on disk as the original Teracrack had done, Flir s program simply produced 19,367 passwords.

Flir considered attempting to get administrative access on the cluster, so he could hide his processes in the future or potentially kick other users jobs and login sessions off the cluster. The idea excited him, having full administrative control of the third fastest publicly known supercomputer in the world. But it was probably unnecessary. He d investigate the feasibility anyway.

He first checked the permissions of directories in /Applications, but they were sound. Either someone had audited the permissions or they hadn t installed any third-party software through dmg files using Finder. The latter seemed very likely. Cluster people were real UNIX-heads and would be unlikely to install software through drag-and-drop. Few people did permissions audits , though maybe the Virginia Tech people had seen the security advisory on this issue. No matter, Flir thought, I still have a heck of a cracking platform!

It would be enough to get the administrative password of one of the administrators on the system. Flir ran nidimp to get a list of users in the administrative group, using grep to get lines where 20, the gid of the staff group, appeared:

 # nidump passwd .  grep :20: yesboss:0NXK4eXxbcrzU:501:20::0:0:Cluster Admin:/Users/yesboss:/bin/bash mike:4iEeI6d1MQKTs:502:20::0:0:Mike:/Users/mike:/bin/tcsh ed:5jGeI8k1MQKTs:503:20::0:0:Ed:/Users/ed:/bin/tcsh bob:sTKeI6d1MQI4e:504:20::0:0:Bob:/Users/bob:/bin/tcsh dave:I8/zwIfZ35jl2:505:20::0:0:Dave:/Users/dave:/bin/tcsh 

He was pleased to see that not only did he get a list of users in the staff group, but also that he got non- shadowed passwords stored in 13-character crypt() format.

He was surprised that the cluster hadn t been upgraded to OS X 10.3, where passwords were hashed with a stronger algorithm. That surprise lasted until he thought about the ramifications of upgrading the operating system on the entire cluster. Outside of the downtime required to upgrade or reinstall and the approximately $43,000 license costs, there was one critical issue. The folks at Virginia Tech had needed to use several kernel-level third-party products. Each of those products would have to be tested on the new operating system update. Those that didn t work would need to be ported. Finally, the new cluster would need to be tested to confirm that performance hadn t taken a hit. This could be accomplished by building a small, possibly 10-node, mini-cluster or could be attempted by trying a second disk image on the larger cluster during planned downtime. It definitely wasn t something to be undertaken lightly.

Whatever the reason, Flir would try to gain root by cracking the five administrative passwords tomorrow. He d use the larger 2.9-trillion-word dictionary, based on the 51 “million-word dictionary and also the lowercase letters and digits dictionary that he d considered earlier. Each account would require about 40 minutes of runtime, for a worse case total of more than 3 hours. Flir estimated that he d probably get a single password in 20-60 minutes, though and decided to limit the exercise to a single hour . He d run the test the next day, though, if he thought it was worth the risk.

Flir decided to take his password store home for the night now.

^[iii] If you re thinking this would take 1-2 years , remember that we can group the 9.75 passwords that share the same salt into one run. (9.77 passwords/hash = 40,000 passwords at Pacific Tech / 4096 hashes)