Accessing Memory Manually

[sig.johnson]

I have a program that reads some data into a complicated nested list, and I would like to be able to access that data across multiple runs of a Lisp rather than having it cleared when the Lisp ends or the drawing (or even AutoCAD) is closed.

I see a couple possibilities:

1. Print the nested list to a file in a way that preserves the hierarchy, something like this. Then, when it needs to be accessed later, use the reverse of Lee's program to read from the file into a nested list.
2. Access a memory location directly to store the binary data that represents the nested list. Then, when it needs to be accessed later, tell my program that there is a nested list stored at that memory location and whatever is in memory there can now be accessed through a list variable in the current Lisp session. This behavior would be reminiscent of a pointer in C.

I would definitely prefer to do option 2, since I think it's a more elegant solution and doesn't require transcribing and reverse-transcribing to a text file. The list I am storing will be very large, so if I have to do option 1, it could result in a pretty sizable text file.

Is option 2 even possible? I wasn't able to find any reference to direct memory access with Lisp, but I am still fairly new. I see vl-registry-read and vl-registry-write, but I would like to be able to store my data in a less-delicate folder.

If not option 2, is anybody aware of an existing program that reads a text file into a nested list?

[john.uhden]

@sig.johnson 

I feel confident that a number of us here could could look at the list structure and figure out how to write and read it to/from a text file.

Could you post a section of the list as an example?  If you have 2500 sub-items all at the same level I think it would suffice to show say 3 of them, etc. etc.

I am imagining this as like a directory tree, which could help our thought process.

[sig.johnson]

This is what the list looks like after printing to a text file using Lee Mac's princf.

Note: this has confidential information removed and only shows a few (of many more) instructions from only the first two (of 100) drawings. I estimate the actual file would be about 10k lines.

(
    (
        ("0.50" "10.90" "MTEXT" nil "edit" nil "TEXT I CAN'T POST ON THE FORUM")
        ("2.00" "9.90" "dest-arrow" "HA4D3" "edit" "SIGCODE" "HTR_LT_L1")
        ("2.00" "9.60" "dest-arrow" "HA4D3" "edit" "SIGCODE" "HTR_LT_L2")
        ("2.00" "9.30" "dest-arrow" "HA4D3" "edit" "SIGCODE" "HTR_LT_L3")
        ("2.00" "9.00" "dest-arrow" "HA4D3" "edit" "SIGCODE" "GND")
    )
    (
        ("0.50" "10.90" "MTEXT" nil "edit" nil "MORE TEXT I CAN'T POST")
        ("4.10" "6.00" "MTEXT" nil "edit" nil "HEATER 01")
        ("4.80" "8.40" "MTEXT" nil "edit" nil "RTD001")
        ("4.80" "7.50" "MTEXT" nil "edit" nil "RTD002")
    )
)

For reference, this is what the structure means. My comments after ";".

(   ; begin full list
    (   ; begin first drawing out of 100
        ("instruction part 1" "instruction part 2" ...)   ; instructions come as lists of strings or nils with 5-8 entries each
        ("instruction part 1" "instruction part 2" ...)
        ...   ; many more instructions per drawing
    )   ; end first drawing
    (   ; begin second drawing
        ("instruction part 1" "instruction part 2" ...)
        ...
    )
    ...   ; many more drawings
) ; end list
    

[john.uhden]

@sig.johnson 

10k lines of that stuff ain't gonna be like lightning, but I see how to read it back into a list.

Maybe I'll test a little with your sample.

[sig.johnson]

Thanks for taking a look!

To clarify, is writing/reading to a text file the easier way to do it because...

(a) there are no ways to directly interact with memory in AutoLisp

or

(b) there are ways to directly access memory, but they are pretty hard, so this is simpler?

[john.uhden]

@sig.johnson 

It's sure as hello simpler for me.  I know nothing about direct memory addressing.

Besides, if you're interrupted in your work, or someone else on your network needs to access the file, there it is, and as plain text that you can read, without any umlauts or happy faces.

We can even add a remarks section at the top.

You just can't have any word wrapping.

[martti.halminen · Accepted solution]

You are trying to do this far too complicated. (Though Autodesk has also somewhat complicated the matter.)

There is no need to create a pretty-printed human-readable storage form, you need something that Lisp can read in with minimal parsing.

Pretty much all the history of Lisp there has been a feature called print-read consistency. It means that except for unreadable objects (like function definitions etc.) you can dump data with PRINT or PRIN1 and calling READ on that creates the same list structure, however heavily nested.

In most Lisp implementations READ can read also from a file, but Autodesk screwed that up: AutoLISP READ does what would be called in Common Lisp READ-FROM-STRING.

So we are reduced to using READ-LINE, which adds the requirement that the whole list must be on a single line, i.e. it can't contain strings with newline characters.

With that limitation this is easy:

_$ (setq aa '(
(
("0.50" "10.90" "MTEXT" nil "edit" nil "TEXT I CAN'T POST ON THE FORUM")
("2.00" "9.90" "dest-arrow" "HA4D3" "edit" "SIGCODE" "HTR_LT_L1")
("2.00" "9.60" "dest-arrow" "HA4D3" "edit" "SIGCODE" "HTR_LT_L2")
("2.00" "9.30" "dest-arrow" "HA4D3" "edit" "SIGCODE" "HTR_LT_L3")
("2.00" "9.00" "dest-arrow" "HA4D3" "edit" "SIGCODE" "GND")
)
(
("0.50" "10.90" "MTEXT" nil "edit" nil "MORE TEXT I CAN'T POST")
("4.10" "6.00" "MTEXT" nil "edit" nil "HEATER 01")
("4.80" "8.40" "MTEXT" nil "edit" nil "RTD001")
("4.80" "7.50" "MTEXT" nil "edit" nil "RTD002")
)
))

_$ (setq f (open "C:/temp/trash.txt" "w"))
#<file "C:/temp/trash.txt">
_$ (prin1 aa f)
((("0.50" "10.90" "MTEXT" nil "edit" nil "TEXT I CAN'T POST ON THE FORUM") ("2.00" "9.90" "dest-arrow" "HA4D3" "edit" "SIGCODE" "HTR_LT_L1") ("2.00" "9.60" "dest-arrow" "HA4D3" "edit" "SIGCODE" "HTR_LT_L2") ("2.00" "9.30" "dest-arrow" "HA4D3" "edit" "SIGCODE" "HTR_LT_L3") ("2.00" "9.00" "dest-arrow" "HA4D3" "edit" "SIGCODE" "GND")) (("0.50" "10.90" "MTEXT" nil "edit" nil "MORE TEXT I CAN'T POST") ("4.10" "6.00" "MTEXT" nil "edit" nil "HEATER 01") ("4.80" "8.40" "MTEXT" nil "edit" nil "RTD001") ("4.80" "7.50" "MTEXT" nil "edit" nil "RTD002")))
_$ (close f)
nil
_$ (setq f2 (open "C:/temp/trash.txt" "r"))
#<file "C:/temp/trash.txt">
_$ (equal aa (read (read-line f2)))
T
_$ (close f2)

- obviously in production use you'd store the READ result in the appropriate variable in another session.

[john.uhden]

@martti.halminen 

Yes, that's the way. 😊

[CodeDing]

@sig.johnson ,

You seem pretty savvy with programming, so I might recommend a SQLite database. User @Anonymous created an awesome module to expose database creation/read/write to AutoLISP a while back. I dabbled with it a year or so back and it works well once you get the hang of it.

The post is over at the swamp:

http://www.theswamp.org/index.php?topic=28286.0 

Best,

~DD

[Sea-Haven]

Like others here the file structure can be simplified even more if writing to a file you have groups of lines that are converted to a list with sublist by the quick look I had.

So just write lines with a delimiter say space or the common one a comma, each group just seperated by a blank line.

I use a Lee-mac function to convert read-line to a list. A simple program to rebuild the array as a list with sublists.

"0.50" "10.90" "MTEXT" nil "edit" nil "TEXT I CAN'T POST ON THE FORUM"
"2.00" "9.90" "dest-arrow" "HA4D3" "edit" "SIGCODE" "HTR_LT_L1"
"2.00" "9.60" "dest-arrow" "HA4D3" "edit" "SIGCODE" "HTR_LT_L2"
"2.00" "9.30" "dest-arrow" "HA4D3" "edit" "SIGCODE" "HTR_LT_L3"
"2.00" "9.00" "dest-arrow" "HA4D3" "edit" "SIGCODE" "GND"

"0.50" "10.90" "MTEXT" nil "edit" nil "MORE TEXT I CAN'T POST"
"4.10" "6.00" "MTEXT" nil "edit" nil "HEATER 01"
"4.80" "8.40" "MTEXT" nil "edit" nil "RTD001"
"4.80" "7.50" "MTEXT" nil "edit" nil "RTD002

[sig.johnson]

Thanks all for the assistance!

I've been able to refactor my program from taking 1 hour to process the 100 sheets to about 16 minutes. Huge quality-of-life improvement.

[Sea-Haven]

I did something took 4hours plus manually 1st go 35 minutes, 3rd version, current 2 minutes, 1 more program change expect say 30% less time again. Inserting 1000+ blocks is like 2 seconds compared to 50 seconds.

If you can identify where all your time delay is maybe a few of us here can help. In working with lists, I am no expert but using mapcar speeds up list manipulation immensely. 

Explain what it is your trying to do overall task.

Just a side comment you can read and write to excel wether its open or not. May be useful.

[sig.johnson]

Based on observations (no actual timing measurements), it appears that the big contributors are (in no particular order)...

• Opening and closing sheets (I have some stuff in my accadoc.lsp file that takes a bit.)
• Executing the modifications that the list entries describe. (This one is pretty inefficient; I have some hacky stuff in here due to my lack of entity-manipulation skill.)
• Signal cross-referencing afterwards (I am using AutoCAD electrical)

Compared to those, the time required for list manipulation is actually pretty small.

I'm relatively happy with where the timing is at right now. I only expect to run this program a handful (<10?) more times during the project lifecycle, so at some point refactoring my code isn't going to be worth the time. But if there's a quick fix then I would certainly be interested to hear it.