Github talanc / vlisp-profiler?

[RandyBenson]

I wrote a pretty good sized lisp project that reads a survey data collector's raw data output and (among other things) creates at least one block insertion in the drawing for each shot recorded by the data collector, inserts utility symbols, tree symbols, and some other things. The inserts are dynamic, attributed blocks with point number, description and elevation attributes.

The program works fine, but it starts out blazing fast and slows down incrementally with each line of input, until at the end of a 300-point run, it's really limping along. Nothing is recursive, and the only thing that gets longer/larger is the list of used point numbers saved and updated with each data point as the variable 'ptnolst'. I can't imagine this is the culprit:

(if (member (atoi ptno) ptnolst)
  nil
  (setq ptnolst (cons (atoi ptno) ptnolst))
)

but if it isn't, I haven't a clue what else could be slowing the program down the longer it runs.

SO - 

I started googling "vlisp program profiler and found a Github project belonging to Thomas Copeland called talanc/vlisp-profiler. It has an MIT license. It hasn't been updated since 2019, but I'm wondering: is anyone here is familiar with it? It purports to run on AutoCAD 2011+.

[daniel_cadext]

Interesting project, not entirely sure how it works, guessing it generates a new lisp file with timers injected.

You could always open an issue on GitHub and ask questions you might have

AutoLISP has been known to barf on larger scales, here’s one example

https://www.theswamp.org/index.php?topic=59487.msg620760#msg620760

[Moshe-A]

@RandyBenson hi,

i can not comment on code i can not see but i can tell you this... autolisp can read data text file as 300 records plus puts these points on graphic screen all in few seconds.  So you need to check your code. One thing to consider is the block insert  (command) function is much slower than (entmakex)  on the other hand inserting block with attributes with (entmake) is a bit harder.

Moshe

[Sea-Haven]

Do you do the stringing of points as well ? Obviously this is a function of CIV3D for me I would use "Civil Site Design" or "Stringer: which have more functions than just read points, they run under CIV3D or Bricscad.

[RandyBenson]

Hi Daniel, thanks for your thoughts. I don't think distance from 0,0 is the issue, 99% of my usual job coordinates are within 2500 feet of 5000,5000. 

[RandyBenson]

Hi Moshe,

Thanks for your comments. Yeah, I don't think it's the size of the text file; it's reading and parsing one line at a time. And the routine uses entmake to create the block inserts. 

Here's a weird thing I didn't think to mention - after the program starts to slow down, if I zoom in or out using the mouse scroll wheel, it speeds back up! Not to the original blazing fast speed, but noticeably faster. That has me more confused than the original issue.

[RandyBenson]

Hi,

The program only inserts the point inserts and symbols that scaled differently in x than y; all the linework and mono-scaled symbols (trees, manholes, etc) are written out to a survey database file and read into Civil 3D from it. Like I said, everything works perfectly, it just slows down imperceptibly with each line of input, so that large files start to crawl (but inexplicably speed up if I zoom with the scroll wheel!). I'll add some function timers and see if I can pin it down.

[daniel_cadext]

My point was that AutoLISP can become embarrassingly slow compared to its competitors using the exact same algorithm.  Not sure if you noticed the time differences. Your algorithm could be bumping up against the same wall.  You’re on the right track with profiling; you can easily inject timers into your code where you think the problem areas are.

[Moshe-A]

@RandyBenson ,

One more thing popup my mind, maybe you are using reactors or have other running reactors that might interfere?

i think we all here gave our best shots, without seeing the code we can't help more 😀

Moshe

[CodeDing]

@RandyBenson ,

---

@RandyBenson wrote:

(if (member (atoi ptno) ptnolst)
  nil
  (setq ptnolst (cons (atoi ptno) ptnolst))
)

---

Might be worth asking, Do you NEED this function? If your data collector does not output duplicate points, then this check is redundant.

If you DO need this check, then before you loop through point numbers, you could convert them all to integers, then sort them, then all you need to check is if the point PRIOR to your current one is equal, then you can just ignore/discard the current point as you're looping through.

Simple tricks like this can greatly improve AutoLISP performance.

Best,

~DD

[RandyBenson]

Hi Moshe,

The only reactor I use is associated with save/quick save, where I have it save a duplicate dwg to a network drive. It doesn't factor in here. I completely understand about not being able to see the code; I'm thinking about creating a Github project for it and if I do, I'll announce it here. 

[RandyBenson]

That's a great suggestion, and it will be easy to test. I'll loop back here.

[RandyBenson]

You're right but I was surprised how fast it runs at first, so I'm hopeful about tracking this down. Also a few C3D versions ago it was fun seeing the individual points populate in the drawing one at a time but something changed and now there's nothing until a bunch of them appear at once. That was explained to me as a change in Windows/C3D interactions.

[RandyBenson]

I'm not sorting them now; not necessary for the 'member' test.

[Sea-Haven]

You mention different X & Y scales, one of the things we did with cIv3D was trees a spread dia & trunk dia, we inserted a dynamic block the Civ3D import points plus stringing, then we ran a lisp it looked at the description TR6*3 which means a 6m spread and a 300 dia trunk, so the dynamic block would be changed to reflect those values, so it may be better to look at cogo point descriptions and do something with the correct block. Maybe post a dwg and some of the codes being used and what is the expected result. 

[RandyBenson]

I commented that 'member test out and it barely moved the needle in terms of speeding it up. I'll look into what else I can comment out without breaking the program. 

[RandyBenson]

The problem of posting the code is where to begin. Here's the first few lines of the main routine (some of the subroutines have 10 or so too):

; RaW Data Zap Multiple v.1.05 - parses TDS Raw Data files, imports PTBLKs
; and Planimetry Inserts into Dwg
;
; ***
(vl-load-com)
(regapp "WRBPTBLKX")
(if (null dxf) (load "dxf"))
(if (null getpbno) (load "getpbno"))
(if (null up) (load "hp"))
(if (null ins_or_repl_pblk) (load "ins_or_repl_pblk"))
(if (null isnum) (load "isnum"))
(if (null LoadXData) (load "LoadXData"))
(if (null mklyr) (load "mklyr"))
(if (null MkPtBlkSS) (load "PtBlkTools"))
(if (null mkrc) (load "mkrc"))
(if (null mkt) (load "mkt"))
(if (null wtf) (load "wtf"))
(if (null parse_dd) (load "parse_dd"))
(if (null parse_ea) (load "parse_ea"))
(if (null parse_desc_front) (load "parse_desc_front"))
(if (null parse_fp) (load "parse_fp"))
(if (null parse_lyr) (load "parse_lyr"))
(if (null parse_opbp) (load "parse_opbp"))
(if (null parse_ptno) (load "parse_ptno"))
(if (null parse_rect) (load "parse_rect"))
(if (null parse_sp) (load "parse_sp"))
(if (null parse_opbp) (load "parse_opbp"))
(if (null parse_stk) (load "parse_stk"))
(if (null parse_ea) (load "parse_ea"))
(if (null parse_lyr) (load "parse_lyr"))
(if (null parse_sp) (load "parse_sp"))
(if (null sort) (load "msort"))
(if (null swapinch) (load "parse_cm"))
;(if (null hortscrns) (load "horticulturelist"))
(if (null dos_copy) (arxload "y:/DOSLib24x64"))
(if (null lm-DynamicPropertyTools)(load "lm-DynamicPropertyTools"))
(if (null createptblk) (load "ptblktools"))
(if (null rwzm-layr) (load "rwzm-layr"))
(if (null attmod) (load "blktools"))
(if (null getpbno) (load "getpbno"))
(if (null fix-el) (load "fix-el"))
(if (null roundoff) (load "rndels"))
(if (null listptlayers) (load "ptin"))
(if (/= (getenv "UserDomain") "WRBENSON") (exit))

...etc.

[daniel_cadext]

"(if (null wtf) (load "wtf"))"

I betting this module runs the fastest

[ec-cad]

You could try doing garbage collection at end of some of those routines.

(GC)

ECCAD