(I need) LISP - binary tree join polilines a watershed

(I need) LISP - binary tree join polilines a watershed

Anonymous
Not applicable
3,816 Views
33 Replies
Message 1 of 34

(I need) LISP - binary tree join polilines a watershed

Anonymous
Not applicable

Hello All,

 

I need a lisp that creates the main stream of thalweg by polylines (basin - watershed) as the example below:

 

pastedImage.png

 

Someone to help?

0 Likes
Accepted solutions (2)
3,817 Views
33 Replies
Replies (33)
Message 2 of 34

marko_ribar
Advisor
Advisor

Computer can't determine that... Brain can...

Marko Ribar, d.i.a. (graduated engineer of architecture)
0 Likes
Message 3 of 34

Anonymous
Not applicable

I believe that a programming can be done for this solution!

0 Likes
Message 4 of 34

ActivistInvestor
Mentor
Mentor

@marko_ribar wrote:

Computer can't determine that... Brain can...


Rubbish.

 

Identifying the root or trunk nodes of a network (which is what a river and its tributaries can be logically modeled as) is a common problem in computer science and is easily-solvable using established algorithms.

 

The first step is generating a DAG (directed acyclic graph) representing the tributaries (edges in the graph) and the points where they subdivide (nodes in the graph).

 

Then all you have to do is follow a path to the root of the graph, which is the trunk or 'main river'. The node representing the upstream starting point of the river is the one that has no incoming edges. The edges whose only incoming starting node originates at the root represents the main trunk of the river, and everything else is a tributary.

 

Because I say it can be done, doesn't mean I'm going to do it, but it most-certainly is possible.

0 Likes
Message 5 of 34

marko_ribar
Advisor
Advisor

I didn't meant that it's not programmable, but certainly is waste of time to find correct main tree by using computation... Only by looking at complete picture you can much faster find solution then computer can, in that I am convinced... It's only matter of technique to create correct longest branch, but firstly you should pick correct tree... That's what brain serves for, not computer... You can't count on my help though as I think that this kind of problems are waste of time... What would you need longest branch, to find it's length and sort it among bunch of branches... For me this is rubbish and trying to do rubbish is also rubbish even bigger...

Marko Ribar, d.i.a. (graduated engineer of architecture)
0 Likes
Message 6 of 34

Kent1Cooper
Consultant
Consultant

It seems extraordinarily challenging to imagine a way that a routine could determine a route that goes "all the way through" [however that would be defined], from all those itty bitty pieces.  But even if it could, it seems utterly impossible for it to be able to choose between your white route and, for example, some of these yellow alternatives:

Alternatives.PNG

 

I would not  assume that whatever possible continuous route happens to be the longest  is necessarily the "main" stream -- one that's a little shorter than some others could easily be carrying more flow.  Largest tributary area?  You'd have to have all the contours, too, because you can't assume the break point is equidistant between stream routes.  But I don't work in Civil Engineering, so what do I know?

 

Looking a little more closely, I see that some of them are 3D Polylines, with Z-coordinate differences, and those are all  along the white route?  But they follow it from the downstream end only about a third of the way along.  Beyond that point they're all 2D, flat on the XY plane.  If the solution involved Joining, and comparing resulting lengths, the 3D ones would need to be Flattened, anyway -- their ends don't all meet in 3 Dimensions.  But what a process -- say you start with the discharge end, and find the Polylines that meet it at one end, Join one of them and check the length, and Join the other and check the length.  Right away the total length from Joining the "right" one is shorter  than the total length from Joining the "wrong" one, so comparing adding single ones isn't going to work.  You'd have to continue Joining all possible combinations of subsequent connected ones, until dead ends are reached, then back off and check a different next connected one, or some such process.  There are more than 2500  Polylines, and it's hard to imagine the number of permutations that would need to be checked....

Kent Cooper, AIA
0 Likes
Message 7 of 34

ActivistInvestor
Mentor
Mentor

@Anonymous wrote:

Hello All,

 

I need a lisp that creates the main stream of thalweg by polylines (basin - watershed) as the example below:

 

 

 

Someone to help?


Good luck. 

 

This is not a trivial problem to solve, and 'binary trees' are not used in the solution (mainly because in a watershed model, tributaries can splt and then rejoin, forming islands of land, and that can't be modeled with a binary tree).

 

Most capable GIS systems have watershed modeling and analysis functions included in them or available via plug-ins, which is probably what you should be using.

 

Homegrown, roll-yer-own solutions are not something you're going to find, and I wouldn't expect anyone here to provide one for you, free of charge.

 

Here is something that you might want to pursue, but it's not written in LISP: http://homepage.usask.ca/~lwm885/topaz/

 

 

 

 

 

 

0 Likes
Message 8 of 34

ActivistInvestor
Mentor
Mentor

@marko_ribar wrote:

I didn't meant that it's not programmable, but certainly is waste of time to find correct main tree by using computation... Only by looking at complete picture you can much faster find solution then computer can, in that I am convinced... It's only matter of technique to create correct longest branch, but firstly you should pick correct tree... That's what brain serves for, not computer... You can't count on my help though as I think that this kind of problems are waste of time... What would you need longest branch, to find it's length and sort it among bunch of branches... For me this is rubbish and trying to do rubbish is also rubbish even bigger...


 

Well I'm happy to have helped you with your English by teaching you a new word today ('rubbish'), but I don't find your opinion very interesting, and I don't agree with you at all.

 

You don't seem to understand the problem. The OP did not show the problem, the OP showed the solution to the problem, or the desired result. Erase the white polyline representing the desired result, and you will then see the problem.

 

The problem is finding and joining together all of the individual polylines that form the single white polyline representing the trunk branch. In other words, you don't start with that single, long white polyine representing the trunk. You start with a bunch of individual polylines representing it, that must be found and joined together to form one polyline, which is the one colored in white in the OP's drawing.

 

There is no way that a human can solve that problem faster than a computer, considering that there could be hundreds-to-thousands of pieces of the trunk that must be found and joined together to form a single entity that represents it.

 

 

 

0 Likes
Message 9 of 34

Anonymous
Not applicable

@Kent1Cooper I forgot to convert the initial polylines of the thalweg to 2d really are erroneously in 3D.

0 Likes
Message 10 of 34

ActivistInvestor
Mentor
Mentor

@Anonymous wrote:

@Kent1Cooper I forgot to convert the initial polylines of the thalweg to 2d really are erroneously in 3D.


Without riverbed elevations, the problem cannot be solved.

 

There are quite a few algorithms that exist for doing this, but all of them rely on the elevation data of the thalweg.

 

Usually, the branch chosen is the one with the lowest mean elevation over its range. Of course, there are other factors that determine weighting of each branch, that could also be involved, but you are removing the information needed to deduce a result, and they don't have to be 2D polylines, because any decent algorithm doesn't rely on what form the data exists in anyways, so long as the needed data is there.

 

Here is some discussion on the subject.

0 Likes
Message 11 of 34

Anonymous
Not applicable

@Kent1Cooper I understood !

Follow the 3D file !

0 Likes
Message 12 of 34

Kent1Cooper
Consultant
Consultant

It's hard to imagine how something like an AutoLisp routine could solve something like this except  by going through the countless  permutations of Joining every possible combination of 3DPolylines, and comparing resulting lengths [assuming that's really a viable basis for choosing the "main" route, about which I have my doubts].  But even if one were to try that, right away, coming in at the downstream end, the very first place where that end-most Polyline has a choice to make between which of two others to Join to, the ends of the three of them are all at different Z coordinates, so they can't be Joined.  Other meeting points I looked at [just a few] did appear to have common Z coordinates, but there are over 2700 Polylines, and I wouldn't assume the only mis-alignment is at that one first spot I looked at.

 

The whole thing is way beyond my ability to imagine any AutoLisp-based solution.  One's brain can see pretty quickly what the longer combinations are -- maybe a very few need to be tried and compared to determine the longest, if that's the criterion -- because it can easily ignore the vast majority  of the Polylines that obviously aren't contributing to that.  AutoLisp would have no way to determine which can be ignored except by slogging through all  of them to see whether they lead anywhere.

Kent Cooper, AIA
0 Likes
Message 13 of 34

ActivistInvestor
Mentor
Mentor

@Kent1Cooper wrote:

It's hard to imagine how something like an AutoLisp routine could solve something like this except  by going through the countless  permutations of Joining every possible combination of 3DPolylines, and comparing resulting lengths [assuming that's really a viable basis for choosing the "main" route, about which I have my doubts]. 


Calculating the length of a series of interconnected polylines doesn't require them to be joined. Typically in processes like this, the length of each polyline would be one of the things that would be cached in memory, and the operation itself would be entirely memory-based, without directly involving any polylines at all.

 

 

0 Likes
Message 14 of 34

Kent1Cooper
Consultant
Consultant

@ActivistInvestor wrote:
Calculating the length of a series of interconnected polylines doesn't require them to be joined. Typically in processes like this, the length of each polyline would be one of the things that would be cached in memory, and the operation itself would be entirely memory-based, without directly involving any polylines at all.

Sure, once you know which ones are interconnected.  Wouldn't you first have to compare both endpoints of every Polyline to both endpoints of every other one, in order to determine which could  form possible stream routes?  I imagine only by doing that could you determine which combinations of lengths of unjoined ones to add together to test lengths of possible routes.  Let's see -- 2700+ Polylines times 2 ends raised to the power of.....  I suppose it could be partially limited if they're in 3D, by comparing the higher-Z-coordinate end of every one to the lower-Z-coordinate end of every other, since that's where any flow-direction-relevant connections would be.

Kent Cooper, AIA
0 Likes
Message 15 of 34

ActivistInvestor
Mentor
Mentor

@Kent1Cooper wrote:

 

Sure, once you know which ones are interconnected.  Wouldn't you first have to compare both endpoints of every Polyline to both endpoints of every other one, in order to determine which could  form possible stream routes?  


I don't think that would be needed, because when you're building a network, you would only look for incoming or outgoing edges, but not both (finding the outgoing edges of one node, implicitly finds the incoming edges of the nodes at the other end of the outgoing edges).

 

Have a look at the sample, and you'll see that all polylines have a downstream direction (the start point is always 'upstream' of the end point). Since all the polylines have that same direction, you only have to look at/compare all the start points or all the end points, depending on the algorithim. So for example, to find all of the incoming edges for a given node (coming from upstream nodes), you would only have to compare the end point of each nearby polyline with the point for the node whose incoming edges you are searching for.

 

Most commercial watershed modeling solutions require their input geometry to have that same upstream-to-downstream direction constraint. I'll also guess that the OPs data is coming from a GIS system, but he's still dreaming if he thinks someone is going to build the solution for him, because that's a lot of work.

 

 

 

 

0 Likes
Message 16 of 34

marko_ribar
Advisor
Advisor

Hi... Me again... Although I thought I won't post code for this, I had some spare time to code my poor attempt... It's brute force recursions version and I couldn't debugg it as it won't pass even first recursion loopings to gather all entities that belong to the tree - in posted DWG - all blue polylines... Like I said this is way beyond everyone's computer capabilities and like I stated - instead of coding for an algorithm, you can probably manually join correct array of polylines into single correct one in no time in comparison to dummy computer... Anyway here is it - recursion hard error... BTW. Prepare DWG like you should only 2D, remove boundary - red polyline and your solution... Finally pick single OUTER branch or one DEAD END branch...

 

(defun c:processtree ( / treechain trunkconst processreverse c r rr treeentities outerbranches trunk trunks dmax s trunkmax ) ;;; variable sss is global

  (vl-load-com)

  (defun treechain ( c / sp ep rtn ) ;;; r is global
    (setq sp (vlax-curve-getstartpoint c))
    (setq ep (vlax-curve-getendpoint c))
    (setq rtn (append (vl-remove-if 'listp (mapcar 'cadr (ssnamex (ssget "_C" sp sp)))) (vl-remove-if 'listp (mapcar 'cadr (ssnamex (ssget "_C" ep ep))))))
    (foreach e rtn
      (if (not (vl-position e r))
        (setq r (cons e r))
      )
    )
    (foreach e rtn
      (treechain e)
    )
  )

  (defun trunkconst ( c / sp ep rtn nextc ) ;;; r is global
    (setq sp (vlax-curve-getstartpoint c))
    (setq ep (vlax-curve-getendpoint c))
    (cond
      ( (and (car r) (ssmemb (car r) (ssget "_C" sp sp)))
        (setq rtn (vl-remove-if 'listp (mapcar 'cadr (ssnamex (ssget "_C" ep ep)))))
      )
      ( (and (car r) (ssmemb (car r) (ssget "_C" ep ep)))
        (setq rtn (vl-remove-if 'listp (mapcar 'cadr (ssnamex (ssget "_C" sp sp)))))
      )
      ( t
        (setq rtn (append (vl-remove-if 'listp (mapcar 'cadr (ssnamex (ssget "_C" sp sp)))) (vl-remove-if 'listp (mapcar 'cadr (ssnamex (ssget "_C" ep ep))))))
      )
    )
    (setq nextc (car (vl-sort (vl-remove-if '(lambda ( x ) (and (vl-position x r) (vl-position x rr))) (vl-remove c rtn)) '(lambda ( a b ) (< (vlax-curve-getdistatparam a (vlax-curve-getendparam a)) (vlax-curve-getdistatparam b (vlax-curve-getendparam b)))))))
    (if (not (vl-position c r))
      (setq r (cons c r))
    )
    (if nextc
      (trunkconst nextc)
    )
  )

  (defun processreverse ( c / sp ep rtn nextc ) ;;; rr is global
    (setq sp (vlax-curve-getstartpoint c))
    (setq ep (vlax-curve-getendpoint c))
    (cond
      ( (and (car rr) (ssmemb (car rr) (ssget "_C" sp sp)))
        (setq rtn (vl-remove-if 'listp (mapcar 'cadr (ssnamex (ssget "_C" ep ep)))))
      )
      ( (and (car rr) (ssmemb (car rr) (ssget "_C" ep ep)))
        (setq rtn (vl-remove-if 'listp (mapcar 'cadr (ssnamex (ssget "_C" sp sp)))))
      )
      ( t
        (setq rtn (append (vl-remove-if 'listp (mapcar 'cadr (ssnamex (ssget "_C" sp sp)))) (vl-remove-if 'listp (mapcar 'cadr (ssnamex (ssget "_C" ep ep))))))
      )
    )
    (setq nextc (car (vl-sort (vl-remove-if '(lambda ( x ) (vl-position x rr)) (vl-remove c rtn)) '(lambda ( a b ) (< (vlax-curve-getdistatparam a (vlax-curve-getendparam a)) (vlax-curve-getdistatparam b (vlax-curve-getendparam b)))))))
    (if (not (vl-position c rr))
      (setq rr (cons c rr))
    )
    (if (vl-position nextc (apply 'append trunks))
      (processreverse nextc)
    )
  )

  (while (or (not (setq c (car (entsel "\nPick tree outer branch curve entity...")))) (if c (vl-catch-all-error-p (vl-catch-all-apply 'vlax-curve-getstartpoint (list c)))))
    (prompt "\nMissed or picked wrong entity type...")
  )
  (treechain c)
  (setq treeentities r)
  (setq r nil)
  (setq outerbranches (vl-remove-if '(lambda ( x ) (and (vl-some '(lambda ( y ) (vlax-curve-getparamatpoint y (vlax-curve-getstartpoint x))) (vl-remove x treeentities)) (vl-some '(lambda ( y ) (vlax-curve-getparamatpoint y (vlax-curve-getendpoint x))) (vl-remove x treeentities)))) treeentities))
  (foreach branch outerbranches
    (while (not (vl-every '(lambda ( x ) (vl-position x (apply 'append trunks))) treeentities))
      (trunkconst branch)
      (setq trunk r)
      (setq trunks (cons trunk trunks))
      (processreverse (car trunk))
      (setq r nil)
    )
    (if (null dmax)
      (setq dmax 0.0)
    )
    (foreach trunk trunks
      (setq d (apply '+ (mapcar '(lambda ( x ) (vlax-curve-getdistatparam x (vlax-curve-getendparam x))) trunk)))
      (if (> d dmax)
        (setq dmax d trunkmax trunk)
      )
    )
    (setq trunks nil rr nil)
  )
  (setq sss (ssadd))
  (foreach c trunkmax
    (ssadd c sss)
  )
  (prompt "\nHighlighted trunk of maximum length... Sel.set is stored in variable \"sss\"...")
  (sssetfirst nil sss)
  (princ)
)

HTH., M.R.

But I doubt that you'll profit of it...

At least I could perhaps get kudo when solution is beyond my capabilities...

Marko Ribar, d.i.a. (graduated engineer of architecture)
Message 17 of 34

marko_ribar
Advisor
Advisor

Here, I revised my code and it works... Slooow, but it works...

 

(defun c:processtree ( / treechain trunkconst processreverse c r rr treeentities outerbranches trunk trunks dmax d trunkmax ) ;;; variable sss is global

  (vl-load-com)

  (defun treechain ( c / sp ep rtn ) ;;; r is global
    (setq sp (vlax-curve-getstartpoint c))
    (setq ep (vlax-curve-getendpoint c))
    (setq rtn (append (vl-remove-if 'listp (mapcar 'cadr (ssnamex (ssget "_C" sp sp)))) (vl-remove-if 'listp (mapcar 'cadr (ssnamex (ssget "_C" ep ep))))))
    (foreach e rtn
      (if (not (vl-position e r))
        (progn
          (setq r (cons e r))
          (if (not (eq e c))
            (treechain e)
          )
        )
      )
    )
  )

  (defun trunkconst ( c / sp ep rtn nextc ) ;;; r is global
    (setq sp (vlax-curve-getstartpoint c))
    (setq ep (vlax-curve-getendpoint c))
    (cond
      ( (and (car r) (ssmemb (car r) (ssget "_C" sp sp)))
        (setq rtn (vl-remove-if 'listp (mapcar 'cadr (ssnamex (ssget "_C" ep ep)))))
      )
      ( (and (car r) (ssmemb (car r) (ssget "_C" ep ep)))
        (setq rtn (vl-remove-if 'listp (mapcar 'cadr (ssnamex (ssget "_C" sp sp)))))
      )
      ( t
        (setq rtn (append (vl-remove-if 'listp (mapcar 'cadr (ssnamex (ssget "_C" sp sp)))) (vl-remove-if 'listp (mapcar 'cadr (ssnamex (ssget "_C" ep ep))))))
      )
    )
    (setq nextc (car (vl-sort (vl-remove-if '(lambda ( x ) (or (vl-position x r) (vl-position x rr))) (vl-remove c rtn)) '(lambda ( a b ) (< (vlax-curve-getdistatparam a (vlax-curve-getendparam a)) (vlax-curve-getdistatparam b (vlax-curve-getendparam b)))))))
    (if (not (vl-position c r))
      (setq r (cons c r))
    )
    (if nextc
      (trunkconst nextc)
    )
  )

  (defun processreverse ( c / sp ep rtn nextc ) ;;; rr is global
    (setq sp (vlax-curve-getstartpoint c))
    (setq ep (vlax-curve-getendpoint c))
    (cond
      ( (and (car rr) (ssmemb (car rr) (ssget "_C" sp sp)))
        (setq rtn (vl-remove-if 'listp (mapcar 'cadr (ssnamex (ssget "_C" ep ep)))))
      )
      ( (and (car rr) (ssmemb (car rr) (ssget "_C" ep ep)))
        (setq rtn (vl-remove-if 'listp (mapcar 'cadr (ssnamex (ssget "_C" sp sp)))))
      )
      ( t
        (setq rtn (append (vl-remove-if 'listp (mapcar 'cadr (ssnamex (ssget "_C" sp sp)))) (vl-remove-if 'listp (mapcar 'cadr (ssnamex (ssget "_C" ep ep))))))
      )
    )
    (setq nextc (car (vl-sort (vl-remove-if '(lambda ( x ) (or (vl-position x r) (vl-position x rr))) (vl-remove c rtn)) '(lambda ( a b ) (< (vlax-curve-getdistatparam a (vlax-curve-getendparam a)) (vlax-curve-getdistatparam b (vlax-curve-getendparam b)))))))
    (if (not (vl-position c rr))
      (setq rr (cons c rr))
    )
    (if (vl-position nextc (apply 'append trunks))
      (processreverse nextc)
    )
  )

  (while (or (not (setq c (car (entsel "\nPick tree outer branch curve entity...")))) (if c (vl-catch-all-error-p (vl-catch-all-apply 'vlax-curve-getstartpoint (list c)))))
    (prompt "\nMissed or picked wrong entity type...")
  )
  (treechain c)
  (setq treeentities r)
  (setq r nil)
  (setq outerbranches (vl-remove-if '(lambda ( x ) (and (vl-some '(lambda ( y ) (vlax-curve-getparamatpoint y (vlax-curve-getstartpoint x))) (vl-remove x treeentities)) (vl-some '(lambda ( y ) (vlax-curve-getparamatpoint y (vlax-curve-getendpoint x))) (vl-remove x treeentities)))) treeentities))
  (foreach branch outerbranches
    (while (not (vl-every '(lambda ( x ) (vl-position x (apply 'append trunks))) treeentities))
      (trunkconst branch)
      (setq trunk r)
      (setq trunks (cons trunk trunks)) (last trunks)
      (processreverse (car trunk))
      (setq r nil)
    )
    (if (null dmax)
      (setq dmax 0.0)
    )
    (foreach trunk trunks
      (setq d (apply '+ (mapcar '(lambda ( x ) (vlax-curve-getdistatparam x (vlax-curve-getendparam x))) trunk)))
      (if (> d dmax)
        (setq dmax d trunkmax trunk)
      )
    )
    (setq trunks nil rr nil)
  )
  (setq sss (ssadd))
  (foreach c trunkmax
    (ssadd c sss)
  )
  (prompt "\nHighlighted trunk of maximum length... Sel.set is stored in variable \"sss\"...")
  (sssetfirst nil sss)
  (princ)
)

HTH., M.R.

Now it's your turn to mark solution, or give a kudo...

Marko Ribar, d.i.a. (graduated engineer of architecture)
Message 18 of 34

MunteanStefan
Contributor
Contributor

@marko_ribar wrote:

Here, I revised my code and it works... Slooow, but it works...



Marko, I think repeated selection sets makes it slow.

Here is a demo, all the objects are processed from a single selection set. The lisp version takes 30 sec., the compiled version only 15 sec.

(anyway, the video is edited, for the entire area some of the waiting time is cut off)

 

The lisp is unfinished, so I'm not going to post it yet.

The algorithm is pretty simple, in each step retain the last segments and add them to all branches (from the previous step) for which there is a continuity, eliminate the shortest branches, eliminate all from the initial list, repeat...

Because every polyline direction is downstream, you only need to search if the start point of a polyline is the end point of any other remaining polylines. If not, it is an end segment.

There are some issues with this algorithm. If an object is duplicated, it is never found as the last segment and the lisp ends in an infinite loop.

 

 

 

 

Message 19 of 34

dbroad
Mentor
Mentor

I hate to say it after so much unpaid effort on your part, but providing these kinds of solutions is exactly what is turning this newsgroup into a forum for moochers rather than for programmers.  Here are a few links of interest:

 

Rush - Something For Nothing

 

People want something for nothing

Architect, Registered NC, VA, SC, & GA.
0 Likes
Message 20 of 34

dgorsman
Consultant
Consultant

@Kent1Cooper - that's why I tend to go for hybrid solutions in cases like this.  Use the computer for the stuff it's good at, use the human for the stuff they're good at, so when the tool is being run by the end user it's both fast and reliable.

 

The actual processing time *can* be brought down a bit but that requires some "data gymnastics" to preprocess the data into logical chunks, much like prepping points for triangulation.  And a whole lotta list functions.  Smiley Very Happy

----------------------------------
If you are going to fly by the seat of your pants, expect friction burns.
"I don't know" is the beginning of knowledge, not the end.


0 Likes