Announcements

Between mid-October and November, the content on AREA will be relocated to the Autodesk Community M&E Hub and the Autodesk Community Gallery. Learn more HERE.

Reading data from txt file II: Log files

Reading data from txt file II: Log files

Anonymous
Not applicable
512 Views
10 Replies
Message 1 of 11

Reading data from txt file II: Log files

Anonymous
Not applicable
I typically waste 2-3 days trying to figure something out before asking for help (I should rethink this...)...

I'm beginning a project to visualize webpage logs. First up: IP addresses.

I read in a line from the log which gives me an array of an IP address where:
ip = "1"
ip = "3"
ip = "0"
ip = "."

and so on to include all 4 octets of the IP address.

I'm looking along the lines of

for i in 1 to ip.count do...

Goals:

- create an array of unique IPs (i.e. prune off all duplicates)
- can you create an array of arrays? i.e. an array where ArrayA = (130, 203, 135, 83), ArrayA = (66, 249, 72, 196), etc.
- create an array where Array = (130, 203, 135, 83, X, T) where X was the number of duplicates and T is the time the webpage was viewed (which is a field in the log)

There's more, but I'm hoping if I get up to speed w/ these, I can slog on my own from there.
0 Likes
513 Views
10 Replies
Replies (10)
Message 2 of 11

Steve_Curley
Mentor
Mentor
Firstly, there may be better (faster) ways to read the data - any MXS gurus feel free to jump in here...

Seeing as you want to sort and do testing on the IP, a number is quicker than text (to compare), and as you can see from the results, the 1st and 3rd lines wouldn't match a text compare, but they would match as numerics.


(
local IPArray = #(#())
IPArray.Count = 0

f = openFile "c:\\the_web.log"
if f != undefined then
(
while (not eof f) do
(
ss = readLine f as stringStream
IP = readDelimitedString ss " " -- IP address as text
Tim = readDelimitedString ss " " -- time as string
ss1 = IP as stringStream
IP1 = readDelimitedString ss1 "."
IP2 = readDelimitedString ss1 "."
IP3 = readDelimitedString ss1 "."
IP4 = readDelimitedString ss1 " "
nIP = (IP1 as integer) * 16777216
nIP = nIP + ((IP2 as integer) * 65536)
nIP = nIP + ((IP3 as integer) * 256)
nIP = nIP + (IP4 as integer)
Append IPArray #(nIP, IP, Tim)
)
close f
for t = 1 to IPArray.Count do
(
format "NumericIP = %, Text IP = %, Time = %\n" IPArray IPArray IPArray
)
)
)


Max 2016 (SP1/EXT1)
Win7Pro x64 (SP1). i5-3570K @ 4.4GHz, 8Gb Ram, DX11.
nVidia GTX760 (2GB) (Driver 430.86).

0 Likes
Message 3 of 11

Anonymous
Not applicable
Thanks; I think this is going to give me a lot of tools/understanding to go fwd from here. A few questions:

- when you use readDelimitedString "."(or readValue), it's basically "capture everything until you find this character (dot, comma, etc)". Yes?
- then for the next read, it starts from where you left off, it doesn't go back to beginning of line, yes?

- what's going on w/ the nIP section where you use 16777216, 65536, 256?

- format: IPArray... don't understand the

- the BIG question: where can I go thru tutorials/courses that teach all this? All this file-reading/Stringstream/filestream content... reading the MXScript help doesn't seem to be working for me. I've went thru Bobo's tuts a couple times, but I dont' believe they cover these specifics.

Thanks again for your help.
0 Likes
Message 4 of 11

Steve_Curley
Mentor
Mentor
Thanks; I think this is going to give me a lot of tools/understanding to go fwd from here. A few questions:

Fire away 🙂

- when you use readDelimitedString "."(or readValue), it's basically "capture everything until you find this character (dot, comma, etc)". Yes?
- then for the next read, it starts from where you left off, it doesn't go back to beginning of line, yes?

Yes and Yes. Essentially "read until you find the delimiter" then "throw away the delimiter". If the values in the string were not an IP address you could use readValue instead of the readDelimitedString, but that won't work in this instance because of the extra "dots" - it tries to work, but fails because a number can't have more than 1 decimal point (the dots).

- what's going on w/ the nIP section where you use 16777216, 65536, 256?

An IP address can be looked at as a 32bit integer value. Each octet can have one of 256 values (0-255)., so each successive octet (from right to left) is a factor of 256 greater than the preceding one.

0.0.0.254 = 254
0.0.0.255 = 255
0.0.1.0 = 256
0.0.1.1 = 257
0.0.255.255 = (255*256)+255 = 65535
0.1.0.0 = 65536
and so on. Basically Hexadecimal notation but using decimal digits. All it is doing is converting the textual IP address into a number.

- format: IPArray... don't understand the

Think of it as a spreadsheet - rows and columns
.              C O L U M N S

| 1 | 2 | 3 |
--|-----------|-----------------|-------|
R 1 | 16909060 | 1.2.3.4 | 00:01 |
--|-----------|-----------------|-------|
O 2 | 84281096 | 5.6.7.8 | 01:30 |
--|-----------|-----------------|-------|
W 3 | 16909060 | 001.002.003.004 | 13:50 |
--|-----------|-----------------|-------|
S 4 | 134810123 | 8.9.10.11 | 16:00 |
--|-----------|-----------------|-------|

IPArray = the whole of the 1st row, IPArray = the contents of the 1st column in the 1st row.
The variable "t" is looping from 1 to 4 thus selecting each row in turn.

- the BIG question: where can I go thru tutorials/courses that teach all this? All this file-reading/Stringstream/filestream content... reading the MXScript help doesn't seem to be working for me. I've went thru Bobo's tuts a couple times, but I dont' believe they cover these specifics.

Yep - that's the biggy.
The principles are common to most languages, it's the syntax which is different - that's where the MXS help comes in. There's a couple of books on MXS (though I don't have any and so can't recommend which to get) which may be of some help. The MXS help does cover them (readStringDelimited for example) but in a fairly basic "this is what it does" kind of fashion - there are some examples though, so it's a case of extrapolating from the examples given to get what is required in each specific case.

Thanks again for your help.

You're welcome 🙂

Max 2016 (SP1/EXT1)
Win7Pro x64 (SP1). i5-3570K @ 4.4GHz, 8Gb Ram, DX11.
nVidia GTX760 (2GB) (Driver 430.86).

0 Likes
Message 5 of 11

Anonymous
Not applicable
I typically waste 2-3 days trying to figure something out before asking for help (I should rethink this...)...

I'm beginning a project to visualize webpage logs. First up: IP addresses.


Interesting 3DS Max script project. Almost seems as if you are developing a tool and using 3DS Max to compile/run your script for something not 3D related. Just curious.

Mike
0 Likes
Message 6 of 11

Anonymous
Not applicable
I've had a pretty productive (and fun) morning by employing your pointers. Guess I'll look for a book, cause I feel like I'm just not learning by reading Help and referring to examples. As you said... "principles... common to most languages"... this is where I need work.

Thanks again.
0 Likes
Message 7 of 11

Steve_Curley
Mentor
Mentor
You could try some other languages - PHP, Perl, Python etc. Many more books on those languages than on MXS. There are free C/C++ and Delphi (object Pascal) compilers too - the choice is endless.

Max 2016 (SP1/EXT1)
Win7Pro x64 (SP1). i5-3570K @ 4.4GHz, 8Gb Ram, DX11.
nVidia GTX760 (2GB) (Driver 430.86).

0 Likes
Message 8 of 11

Anonymous
Not applicable
using 3DS Max to compile/run your script for something not 3D related.


It's a data visualization project aimed at network security... filtering data to help analyze current situations and eventually provide best options for decision-makers. There's a few different tacks: 2d Flash, 3d, 3d immersive and sonification, and I'm digging into the 3d side of things.
0 Likes
Message 9 of 11

Anonymous
Not applicable
There are free C/C++ ... compilers too - the choice is endless.


This seems like the smartest course of action. I'll delve into this; thanks.
0 Likes
Message 10 of 11

Anonymous
Not applicable
I believe these are all free downloads:
http://www.microsoft.com/express/product/
0 Likes
Message 11 of 11

Anonymous
Not applicable
Cool; thanks Mike.
0 Likes