Fun with Python - Log Analysis with Regex v2
Introduction
In the first installment of FwP, I took a look at using Python and regex for log analysis. The code read a text file, searched it for IPv4 addresses, and printed the results to screen.
I have extended its functionality to include a CLI prompt and menu, and the ability to write the results to a .csv file in a secure format.
The catalyst for development was the need to pull a list of malicious IPv4 addresses from a log file, and store them for use in a threat feed and blocklist. It will find IPv4 addresses in any text file, but was developed to read a full log file that had been prefiltered by a security device to capture all inbound scans from unknown IPs.
Preparation
If you haven’t done so, go back to the first article and review the code to understand its basic functionality, as this code is a modified version of the original.
This article assumes Python is already installed, and the reader has basic CLI knowledge.
Copy the following text into Notepad and save it as “log.txt”.
apple dog river blue chair 192.168.0.24 star sky orange quiet dog mouse 10.0.45.67 grass key bottle guitar notebook lamp fire dog spoon clock 192.168.0.86 mountain flower tree shadow water dog mirror balloon paper 10.0.122.89 hat snow zebra ocean desert dog light thunder 192.168.0.145 cave rainbow
dog breeze forest pencil moonstone journey whisper dog stone coffee melon dog cactus ice 10.0.250.36 wind breeze drift berry dog meadow rocket path 192.168.0.78 island cloud flame seed ladder canvas dog candle frost anchor twig 10.0.89.14 marble echo vine cherry storm dog river bird fog canvas river bench
dog arrow prism veil 192.168.0.199 jade dream arrow harp nest dog glove cedar honey quilt log 10.0.34.78 stream sail crystal timber scarf glow cave flame spruce clay coral rust dog fog sunrise trunk 192.168.0.123 velvet wheel ember grove amber dog lamp pearl frost bridge lily shadow ripple coin drift
stone dog 192.168.0.36 breeze leaf chair frost 10.0.78.22 wing river anchor bench dog blossom ember path crystal moss glove 192.168.0.144 ladder dusk tree plank dog flame glow veil snow harp cedar twig log stream charm root echo bird dog bark 10.0.13.88 dream
mountain frost dog paper canvas shadow ladder glass lamp sunrise 192.168.0.92 ocean honey coral cedar dusk dog ivy quilt timber rust wave stone flame sail branch breeze fog berry star log moon dog coral spruce arrow ripple fog flame drift 10.0.56.34 nest frost
dog wheel echo shade vine amber stone frost tree 10.0.211.9 whisper hat charm ladder river bark flame pearl dog paper coin drift meadow mirror rust shadow dog 192.168.0.199 ember dusk root star glimmer twig crystal dog bench tide light breeze wing veil ember plank moon dog
The Code
This code is free to use under the terms of the GNU General Public License. I offer no guarantees or warranties. Use at your own risk.
Copy the code into Notepad:
import os
import re
app_running = True
log_file = open("log.txt")
ip_file = open("ip_list.csv", "a")
log_strings = log_file.readlines()
def find_ipv4(n):
ipv4_list = re.findall(r"(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})", str(log_strings))
write_to_csv(ipv4_list, n)
def write_to_csv(ips, n):
unsecured_list = ""
for x in ips:
unsecured_list += (x + ",")
match n:
case 0:
ip_file.write(unsecured_list)
print("Your file of unsecured IPs is being generated. Use with caution.")
app_running = False
case 1:
secured_list = unsecured_list.replace(".","[.]")
ip_file.write(secured_list)
print("Your file of secured IPs is being generated. Use with caution.")
case _:
print("Error: Unknown. How did you get here?")
quit()
def menu():
os.system('cls' if os.name == 'nt' else 'clear')
ui = ""
print("*********** Find Them IPs ***********")
print("1) Secured List ")
print("2) Unsecured List ")
print("q) Quit ")
print("*************************************")
while app_running:
ui = input("_: ")
match ui:
case "q":
app_running == False
break
case "1":
find_ipv4(1)
case "2":
find_ipv4(0)
case _:
print("Error: command not recognized.")
#call the function menu () to initiate the program and retrieve user input
menu()Save it in the same folder as the “log.txt” file, and name it “analyzer_v2.py”. Make sure to change the option Save as type to All Files, or the file will not save as a .py file.
Open a command prompt and navigate to the folder.
Enter the following command to initiate the program:
python analyzer_v2.py The menu should appear.
Make a selection.
Type 1 to generate a secured list of IPv4 addresses. This will surround each . with brackets, and then write the list to a .csv file named “secured_list.csv”. The program will then quit.
Type 2 to generate an unsecured list of IPv4 addresses. This will not add brackets, and will write the list to a .csv file named “unsecured_list.csv”. The program will then quit.
Type q to quite the program.
Remember, this was developed as a way to gather verified malicious IPv4 addresses.
The unsecured list can be used in threat feeds and blocklists. The secured list makes it safe to distribute, or browse manually, without accidentally clicking a malicious link.
Summary
I won’t summarize the code this time. It is very simple and should be easy to understand with some basic knowledge. It can be easily modified for additional functionality. Don’t forget to include GNU General Public License if you modify and redistribute.
Have fun, thanks for reading.
Daily Cuppa
Today’s cuppa is Organic Vanilla Rooibos provided by Equal Exchange.
Sweet and earthy with a hint of creaminess for the ultimate relaxing cup.
If you enjoyed this article, feel free to buy the author a cup of tea.