Parse JSON with AutoLisp / VisualLisp

[CodeDing]

Hello all,

I think I have come up with an acceptable parser for JSON -> Lists. Hopefully.

But let's be honest, it'll probably break eventually haha..

With my preliminary testing on the 3 attached files (i have tested more, only attached 3) it appears to be working normally and as-expected. But I'm hoping I can get feedback / improvements!

This initial version is actually entirely in 'vanilla' Autolisp (no VisualLisp), so any future releases I will just dictate with an identifier at the top (al = AutoLisp / vl = VisualLisp).

Ideally, the final function should NOT include a 'read' function as this one currently does. But for now, this is a step in the right direction I believe.

It's definitely pretty slow at the moment, but it can handle its own.

The main reason I looked into this was after realizing that the (read) function cannot handle 'symbols' over ~2000 characters (so, one long continuous string [as found in the attached 'Example JSON Google API'] will cause my original (much easier) function to fail:

(defun JSON->LIST (json / tmp dbl nwl)
;json - string, as json data
;returns - list or nil, list of data converted from json
(if (eq 'STR (type json)) (read (vl-string-translate "[]{}:," "()()  " json)) nil)
);defun

Here's the function.. Let me know what you think:

(defun ParseJSON->List (json / cnt len prevChar ch endPos tmp ret)
;Coded by Denon Deterding (Version al:1.0 - 20200221)
;json - string, of json data to convert
;returns - list, of converted json
;helper function(s)
  (defun EndOfQuote (str pos / ch cnt prevChar found)
    (setq cnt 0 prevChar "" found nil)
    (while (not found)
      (setq ch (substr str (+ pos (setq cnt (1+ cnt))) 1))
      (if (and (eq "\"" ch) (not (eq "\\" prevChar))) (setq found t pos (+ cnt pos)))
      (setq prevChar ch)
    );while
    pos
  );defun
  (defun EndOfBracket (str pos openChar / ch cnt prevChar found inStr closeChar)
    (setq cnt -1 prevChar "" found nil inStr nil closeChar (if (eq "{" openChar) "}" "]"))
    (setq openCnt 0 closeCnt 0)
    (while (not found)
      (setq ch (substr str (+ pos (setq cnt (1+ cnt))) 1))
      (if (and (eq "\"" ch) (not (eq "\\" prevChar))) (setq inStr (not inStr)))
      (cond
	((and (not inStr) (eq openChar ch)) (setq openCnt (1+ openCnt)))
	((and (not inStr) (eq closeChar ch)) (setq closeCnt (1+ closeCnt)))
      );cond
      (if (= openCnt closeCnt) (setq found t pos (+ pos cnt)))
      (setq prevChar ch)
    );while
    pos
  );defun
;main work
(setq cnt 0 str "" ret '())
(repeat (setq len (strlen json))
  (setq ch (substr json (setq cnt (1+ cnt)) 1))
  (cond
    ((or (eq "{" ch) (eq "[" ch))
      (setq endPos (EndOfBracket json cnt ch)
	    cnt (1+ cnt)
	    ret (cons (ParseJSON (substr json cnt (- endPos cnt))) ret)
	    str ""
	    cnt endPos)
    );cond 1
    ((eq "\"" ch)
      (setq endPos (EndOfQuote json cnt)
            cnt (1+ cnt)
	    ret (cons (substr json cnt (- endPos cnt)) ret)
	    str ""
	    cnt endPos)
    );cond 2
    ((or (eq ":" ch) (eq "," ch))
      (setq tmp (read (strcase str)))
      (cond
	((or (eq 'NULL tmp) (eq 'FALSE tmp)) (setq ret (cons nil ret)))
	((eq 'TRUE tmp) (setq ret (cons t ret)))
	((numberp tmp) (setq ret (cons tmp ret)))
	((not (null tmp)) (setq ret (cons (read str) ret)))
      );cond
      (setq str "")
    );cond 3
    ((not (member ch '("\n" "\t" "\b" "\r")))
      (if (or (eq " " ch) (= (1+ len) cnt))
	(progn
	  (setq tmp (read (strcase str)))
	  (cond
	    ((or (eq 'NULL tmp) (eq 'FALSE tmp)) (setq ret (cons nil ret)))
	    ((eq 'TRUE tmp) (setq ret (cons t ret)))
	    ((numberp tmp) (setq ret (cons tmp ret)))
	    ((not (null tmp)) (setq ret (cons (read str) ret)))
	  );cond
	  (setq str "")
	);progn
      ;else
        (setq str (strcat str ch))
      );if
    );cond 4
  );cond
);repeat
(reverse ret)
);defun

PLEASE NOTE:

- The one weird caveat to this function, due to its recursive nature and the way it returns items, is that an extra (list ..) is created around the original data. THIS MEANS that you should ALWAYS surround this function with a (car ..) function to return the correct format. (see below)

An Example call:

(setq data (car (ParseJSON->List jsonString)))

If you were to test an example file:

(defun c:TEST ( / str f txt data)
(setq str "" f (open "c:\\myFolder\\my sub folder\\example json.txt" "r"))
(while (setq txt (read-line f))
  (setq str (strcat str txt "\n"))
);while
(close f)
(if (setq data (car (ParseJSON->List str)))
  (princ data)
);if
(princ)
);defun

Best,

~DD

[CodeDing]

--UPDATE--

This is what I get for changing the name at the last minute!

This line will need to be updated to reflect the correct function name:

	    ret (cons (ParseJSON (substr json cnt (- endPos cnt))) ret)
...to...
	    ret (cons (ParseJSON->List (substr json cnt (- endPos cnt))) ret)

See attached.

[CodeDing]

Never got around to making a better one. Here's an updated take w/ VisualLISP (see attached).

This version is better than the... (read (vl-string-translate "[]{}:," "()() " json)) ...method because the translate method will replace ALL of the input characters (even in json object string), where this new function will not.

A small sample:

(ParseJSON->List "{\"myTag\" : \"MyTagValue\" , \"myNum\" : -123.4}")
...returns...
(("myTag" "MyTagValue") ("myNum" -123.4))

[CodeDing · Accepted solution]

**NEW VERSION**
I have realized that the last update I posted (VL1) has a design flaw. When presented with an array as an initial argument, then the list created would close too early, so in the simple example below only the first set of items ("abc" & "def") would be returned and the ("xyz" & "wvu") items would not be. I have attached an updated version (VL2) which is working as expected.

[
      {
         "abc" : 123,
         "def" : 456
      },
      {
         "xyz" : 987,
         "wvu" : 654
      }
]

Best,
~DD

[Anonymous]

@CodeDing 
I was looking for a solution and bumped into your post.

Its function works very well !! excellent for the initiative.

but you have already dealt with a situation like this:

my .JSON has a great length I get it from the google API and it takes a long time to read everything and convert it into a list.
how do you deal to make this processing faster ?? since I just need the addresses

I attached .json

[CodeDing]

@Anonymous ,

Sadly, I do not believe there is a 'faster' approach using the AutoLISP / Visual LISP method and the Google API.

You will have to Get the web text, convert the json, then search for your data with lisp functions.

Alternatively,

Around the same time of this original post, I reached out to the .NET community to ask if somebody could create an AutoLISP function that could convert the JSON faster (since they have more tools and libraries for such a task). @_gile  was kind enough to help me and created a very useful (json2lisp ...) function (via a C# .dll file).

My post on the forums:

https://forums.autodesk.com/t5/net/use-net-to-convert-json-via-autolisp/m-p/9382709#M65121

His updated function at The Swamp:

https://www.theswamp.org/index.php?topic=55858.0

...the only problem that I had with his function was that it ultimately cannot accept a json array as the initial argument. So, in my free time, I have been creating my own .dll file (as well as learning the C# language) with an updated function, and some other useful ones.. such as: an updated web GET call, a GET call that saves the returned response to a file location (such as images, files, etc), functions for custom dwg properties, and the updated json to lisp parser. I plan to add many more functions and document their uses, I was going to release that file in a new post on the forums one day in the future, but If you are interested, I could give you what I have currently and post what functions are available?

Best,

~DD

[Anonymous]

@CodeDing 

I will test the .dll on the link
I would be very grateful if you could attach or send me messages for me to check.
already grateful