Hello everyone and welcome to my blog!
As we come closer to the end of the course, we started working on our final projects. In this project we are going to work with some open source project, practice compiling project, testing different compiler options, test and analyze performance and much more.
The requirements are the following:
As we come closer to the end of the course, we started working on our final projects. In this project we are going to work with some open source project, practice compiling project, testing different compiler options, test and analyze performance and much more.
The requirements are the following:
- Open source software package includes a CPU-intensive function or method that is implemented in a language that compiles to machine code (such as C, C++, or Assembler).
- Test case that takes at least 4 minutes to run and represents a reasonable use-case for the software.
- We should be able to build project on both x86_64 and AArch64 platforms
Final project consists of three stages:
Link to the official website:
https://www.openwall.com/john/
Link to the Github page:
https://github.com/magnumripper/JohnTheRipper
Background info about my machines:
I have a virtual machine with Ubuntu 64bit, and Raspberry Pi4 that with Manjaro 64bit.
Steps to set up:
Now we can finally test.
So using the above credentials the results where the following:
osakhnatska@ubuntu:~/Downloads/JohnTheRipper-bleeding-jumbo/run$ ./john crack1.txt
Using default input encoding: UTF-8
Loaded 1 password hash (sha512crypt, crypt(3) $6$ [SHA512 256/256 AVX2 4x])
Cost 1 (iteration count) is 5000 for all loaded hashes
Warning: OpenMP is disabled; a non-OpenMP build may be faster
Proceeding with single, rules:Single
Press 'q' or Ctrl-C to abort, almost any other key for status
Warning: Only 2 candidates buffered for the current salt, minimum 8 needed for performance.
Almost done: Processing the remaining buffered candidate passwords, if any.
Proceeding with wordlist:./password.lst
Proceeding with incremental:ASCII
alexas (spo)
1g 0:00:02:23 DONE 3/3 (2020-04-02 12:54) 0.006961g/s 1180p/s 1180c/s 1180C/s alexas..alince
Use the "--show" option to display all of the cracked passwords reliably
Session completed.
So as we can see it run for 3 minutes 33 seconds on my x86 ubuntu 64-bit machine.
[alexa@bumblebee run]$ ./john crack.txt
Using default input encoding: UTF-8
Loaded 1 password hash (sha512crypt, crypt(3) $6$ [SHA512 128/128 ASIMD 2x])
Cost 1 (iteration count) is 5000 for all loaded hashes
Will run 4 OpenMP threads
Proceeding with single, rules:Single
Press 'q' or Ctrl-C to abort, almost any other key for status
Warning: Only 2 candidates buffered for the current salt, minimum 8 needed for performance.
Warning: Only 5 candidates buffered for the current salt, minimum 8 needed for performance.
Almost done: Processing the remaining buffered candidate passwords, if any.
Proceeding with wordlist:./password.lst
Proceeding with incremental:ASCII
alexas (spo)
1g 0:00:04:23 DONE 3/3 (2020-04-02 13:53) 0.003790g/s 643.2p/s 643.2c/s 643.2C/s alexas..alley1
Use the "--show" option to display all of the cracked passwords reliably
Session completed.
So as we can see, on aarch64 Manjaro 64-bit machine it took 4 minutes 23 seconds to run.
We can notice some differences in these values on ubuntu and manjaro respectively:
0.006961g/s 1180p/s 1180c/s 1180C/s
and
0.003790g/s 643.2p/s 643.2c/s 643.2C/s
I don't know what these values account for but I assume it they account for how quickly the software processes the password. I will make some more research into it and try to understand in detail what they account for.
Now let's try to estimate performance using time function.
Timings for the ubuntu
1st run:
real 2m19.542s
user 2m13.565s
sys 0m0.313s
2d run:
real 2m15.053s
user 2m14.080s
sys 0m0.204s
user 16m40.134s
sys 0m0.748s
2d run:
- Building and benchmarking
- Profiling
- Optimization
During this few weeks we've been working on stage 1. Or professor, Chris Tyler, suggested that we choose something of a great interest for us, so it is more exciting for us to work on it. I've chose to work on the famous security tool for password cracking - John the Ripper password cracker.
Link to the official website:
https://www.openwall.com/john/
Link to the Github page:
https://github.com/magnumripper/JohnTheRipper
Background info about my machines:
I have a virtual machine with Ubuntu 64bit, and Raspberry Pi4 that with Manjaro 64bit.
Steps to set up:
- I downloaded 1.9.0-jumbo-1 version in tar.xz on my both Aarc64 and x86, extracted it.
- For my x86 Ubuntu machine I used instructions from https://github.com/magnumripper/JohnTheRipper/blob/bleeding-jumbo/doc/INSTALL-UBUNTU
- For my arm Manjaro machine I used instructions from https://github.com/magnumripper/JohnTheRipper/blob/bleeding-jumbo/doc/INSTALL
- *Tip check that you have installed required dependencies mentioned in the install files before you try to build. I only had to install some required software on Ubuntu. I run " sudo apt-get -y install git build-essential libssl-dev zlib1g-dev "
- Enter the directory into which you extracted the source codedistribution of John. Enter the "src" subdirectory. " cd src "
- To build run " ./configure && make "
- To run enter the directory into which you extracted the source codedistribution of John. Enter the "run" subdirectory. " cd run "
- To test software use command ./john --test. It is predefined testing that will allow to get some information about the overall performance of the software. The results of it are too long so I won't be including it here.
Now let's get to benchmarking and test our software! But first we need to prepare a little bit.
To test john we need a password to crack. There are two options:
- you create another user with password on your system then go get it's hash.
- you just create a file and copy some sample valid credentials.
1 Option
If you choose to go this way you will need to install john app on you system using "sudo apt install john". It will allow to read shadowed password file on your system.
Create sample user "sude adduser spo" then enter password (note it shouldn't be too easy and too hard so our tests don't run too fast or too slow.
When you installed john and created some sample user you can use the following command to get this sample user's hashed password: "sudo unshadow /etc/passwd /etc/shadow > mypasswd" This will create mypasswd file in your current location.
Next, read it and find needed credentials, they will be the last ones "cat mypasswd" copy it to sample file that will store it. I created crack.txt.
Now you can run and test john using meaningful data "./john crack.txt"
2 Option
If you decide that you don't want to spend too much time you may just copy existing credentials and save them in a sample file that will store it.
For example, you may copy the following:
spo:$6$robVWbsE$NnwsowPEEQFvKPdthgW0IzTlHsh5f4G71C7rS6fEU.iPLnYYQh1yqywA2TQpkNHugpKCF9SCwnx4btymZhGc7.:1001:1001:,,,:/home/spo:/bin/bash
*Note if you want to use the same credentials and the same file you wil need to remove the previous results. run "rm john.pot" and continue with the same steps that were mentioned above.
So using the above credentials the results where the following:
osakhnatska@ubuntu:~/Downloads/JohnTheRipper-bleeding-jumbo/run$ ./john crack1.txt
Using default input encoding: UTF-8
Loaded 1 password hash (sha512crypt, crypt(3) $6$ [SHA512 256/256 AVX2 4x])
Cost 1 (iteration count) is 5000 for all loaded hashes
Warning: OpenMP is disabled; a non-OpenMP build may be faster
Proceeding with single, rules:Single
Press 'q' or Ctrl-C to abort, almost any other key for status
Warning: Only 2 candidates buffered for the current salt, minimum 8 needed for performance.
Almost done: Processing the remaining buffered candidate passwords, if any.
Proceeding with wordlist:./password.lst
Proceeding with incremental:ASCII
alexas (spo)
1g 0:00:02:23 DONE 3/3 (2020-04-02 12:54) 0.006961g/s 1180p/s 1180c/s 1180C/s alexas..alince
Use the "--show" option to display all of the cracked passwords reliably
Session completed.
So as we can see it run for 3 minutes 33 seconds on my x86 ubuntu 64-bit machine.
[alexa@bumblebee run]$ ./john crack.txt
Using default input encoding: UTF-8
Loaded 1 password hash (sha512crypt, crypt(3) $6$ [SHA512 128/128 ASIMD 2x])
Cost 1 (iteration count) is 5000 for all loaded hashes
Will run 4 OpenMP threads
Proceeding with single, rules:Single
Press 'q' or Ctrl-C to abort, almost any other key for status
Warning: Only 2 candidates buffered for the current salt, minimum 8 needed for performance.
Warning: Only 5 candidates buffered for the current salt, minimum 8 needed for performance.
Almost done: Processing the remaining buffered candidate passwords, if any.
Proceeding with wordlist:./password.lst
Proceeding with incremental:ASCII
alexas (spo)
1g 0:00:04:23 DONE 3/3 (2020-04-02 13:53) 0.003790g/s 643.2p/s 643.2c/s 643.2C/s alexas..alley1
Use the "--show" option to display all of the cracked passwords reliably
Session completed.
We can notice some differences in these values on ubuntu and manjaro respectively:
0.006961g/s 1180p/s 1180c/s 1180C/s
and
0.003790g/s 643.2p/s 643.2c/s 643.2C/s
I don't know what these values account for but I assume it they account for how quickly the software processes the password. I will make some more research into it and try to understand in detail what they account for.
Now let's try to estimate performance using time function.
Timings for the ubuntu
1st run:
real 2m19.542s
user 2m13.565s
sys 0m0.313s
2d run:
real 2m15.053s
user 2m14.080s
sys 0m0.204s
Timings for the manjaro
1st run:
real 4m17.043s1st run:
user 16m40.134s
sys 0m0.748s
2d run:
real 4m20.742s
user 16m39.146s
sys 0m0.732s
As we can see, timings for aarch64 in comparison to the same timing of my x86 are the following:
Now let's experiment with different compiler options.
I used the instructions described in Install file in here https://github.com/magnumripper/JohnTheRipper/blob/bleeding-jumbo/doc/INSTALL and article about compiler optimization from our course notes https://wiki.cdot.senecacollege.ca/wiki/Compiler_Optimizations
I decided to start with option "-g -O3", so lets see if there is any difference in comparison to the default optimization
ubuntu
1st run
2nd run
real 1m51.680s
user 7m17.906s
sys 0m0.978s
user 18m7.422s
sys 0m1.310s
2nd run
real 4m41.971s
user 18m4.697s
sys 0m0.938s
3d run
real 4m41.277s
user 18m11.964s
sys 0m0.751s
Now let's try option "-g -O1" and see if there is a difference in performance
ubuntu
1st run
real 1m51.048s
user 7m16.463s
sys 0m0.681s
As we can see, the timings are faster for real, are significantly slower for user, and are a lot slower for sys.
manjaro
1st run
real 4m51.856s
user 18m7.422s
sys 0m1.310s
2nd run
real 4m47.088s
user 18m26.944s
sys 0m0.771s
3d run
real 4m56.747s
user 19m1.774s
sys 0m0.873s
As we can see the timings are slower for all three.
To summarize everything up, we can see that there is plenty of room for optimization of John the ripper password cracker. It no doubt runs slower on the arch system. Using compiler optimization features makes a little bit a difference, but I'm going to make further research and investigate what makes it run slower. I will keep you guys posted! Please come back and check again later, to see more updates.
user 16m39.146s
sys 0m0.732s
As we can see, timings for aarch64 in comparison to the same timing of my x86 are the following:
- are around 2 times bigger for the "real" timing,
- around 8 times bigger for the "user" timing,
- and around 2 times bigger for the "sys" timing
Now let's experiment with different compiler options.
I used the instructions described in Install file in here https://github.com/magnumripper/JohnTheRipper/blob/bleeding-jumbo/doc/INSTALL and article about compiler optimization from our course notes https://wiki.cdot.senecacollege.ca/wiki/Compiler_Optimizations
I decided to start with option "-g -O3", so lets see if there is any difference in comparison to the default optimization
- clean everything up using ./configure && make clean
- change configurations and add new compiler options using ./configure CFLAGS="-g -O3"
- build using make
- remove the file that contains the results of our password cracking and run john again
The results for the ubuntu machine are the following:
ubuntu
1st run
real 1m51.006s
user 7m16.476s
sys 0m0.760s
user 7m16.476s
sys 0m0.760s
2nd run
real 1m51.680s
user 7m17.906s
sys 0m0.978s
3d run
real 1m36.354s
user 6m9.457s
sys 0m1.658s
user 6m9.457s
sys 0m1.658s
As we can see when there is a difference, now real runs faster, user slower and sys also slower.
Let's try to do the same on the manjaro-arm 64.
manjaro
1st run
real 4m51.856sLet's try to do the same on the manjaro-arm 64.
manjaro
1st run
user 18m7.422s
sys 0m1.310s
2nd run
real 4m41.971s
user 18m4.697s
sys 0m0.938s
3d run
real 4m41.277s
user 18m11.964s
sys 0m0.751s
We can see by the results that the timings are higher for all three.
Now let's try option "-g -O1" and see if there is a difference in performance
ubuntu
1st run
real 1m51.048s
user 7m16.463s
sys 0m0.681s
2nd run
real 1m51.229s
user 7m17.057s
sys 0m0.768s
3d run
real 1m50.413s
user 7m11.506s
sys 0m1.021s
As we can see, the timings are faster for real, are significantly slower for user, and are a lot slower for sys.
manjaro
1st run
real 4m51.856s
user 18m7.422s
sys 0m1.310s
2nd run
real 4m47.088s
user 18m26.944s
sys 0m0.771s
3d run
real 4m56.747s
user 19m1.774s
sys 0m0.873s
As we can see the timings are slower for all three.
To summarize everything up, we can see that there is plenty of room for optimization of John the ripper password cracker. It no doubt runs slower on the arch system. Using compiler optimization features makes a little bit a difference, but I'm going to make further research and investigate what makes it run slower. I will keep you guys posted! Please come back and check again later, to see more updates.
Comments
Post a Comment