remove duplicates from a list not removing all duplicates

[blogginsF6GT8]

Hello Friends !

very new to AUTOLISP and need some help .. I am using Lee Mac's wonderful LM:Unique function (the iterative version) to remove duplicates from a list. 

For some reason the vl-remove doesn't remove  all the instances of the duplicate.

Here is the list i am sending to the function:

(19.625 19.8125 19.1875 19.375 18.75 18.5625 18.125 18.9375 18.3125 19.8125 19.625 19.8125 19.8125 19.625 19.375 18.9375 19.375 19.1875 18.9375 18.75 18.3125 18.5625 18.125 18.3125 18.5625 19.1875 18.75 19.375 18.9375 18.125 18.3125 17.1875 17.75 17.375 16.8125 16.4375 17.75 17.1875 17.375 17.375 16.8125 16.8125 16.4375 16.4375 16.4375 16.8125 17.1875 17.75 17.375 17.75 18.5625 19.1875 19.375 18.75 18.9375 16.8125 17.1875 19.625 19.8125 17.375 18.125 18.3125 17.75 16.4375 16.25)

Here is the result:

(19.625 19.8125 19.1875 19.375 18.75 18.5625 18.125 18.9375 18.3125 18.5625 18.75 18.9375 17.1875 17.75 17.375 16.8125 16.4375 16.25)

And after sorting the results:

(19.8125 19.625 19.375 19.1875 18.9375 18.9375 18.75 18.75 18.5625 18.5625 18.3125 18.125 17.75 17.375 17.1875 16.8125 16.4375 16.25)

As you can see there are 3 instances where the duplication remains.

I just cant figure out why vl-remove sees them as different.

Any help would be greatly appreciated.

[phanaem]

There should be fuzz factor in the lisp function.

Try to increase it's value. Look for something like (equal ... 1e-8) and change it to 1e-5 or 1e-4.

Here is a function that works with your sample:

(defun uniq (l)
  (if
    l
    (cons
      (car l)
      (uniq
        (vl-remove-if
         '(lambda (x)
            (equal (car l) x 1e-5)
          )
          l
        )
      )
    )
  )
)

_$ (setq r (vl-sort (uniq l) '>))
(19.8125 19.625 19.375 19.1875 18.9375 18.75 18.5625 18.3125 18.125 17.75 17.375 17.1875 16.8125 16.4375 16.25)
_$ 

[marko_ribar]

Here is iterative with some fuzz...

  (defun unique ( lst / a ll )
    (while (setq a (car lst))
      (if (vl-some (function (lambda ( x ) (equal x a 1e-6))) (cdr lst))
        (setq ll (cons a ll) lst (vl-remove-if (function (lambda ( x ) (equal x a 1e-6))) (cdr lst)))
        (setq ll (cons a ll) lst (cdr lst))
      )
    )
    (reverse ll)
  )

[paullimapa]

lots of solutions listed here back in 2014:

https://forums.autodesk.com/t5/visual-lisp-autolisp-and-general/removing-duplicates-from-a-list/td-p...

[john.kaulB9QW2]

Removing duplicates from a list is nice beginner challenge (BONUS: there should be 100's of examples to glean from) and I believe I may have even created beginner challenge, or two, on the topic at theswamp.org if you find yourself with a need to learn more techniques/methods.

Back to the topic of duplicate removal, this code from my friend Michael has some really nice lessons contained within (BONUS: it is screamingly fast--speed is typically relative with this topic-!).

Composed and Posted by MP:

(defun RemoveDuplicates ( lst / foo temp )
    (defun foo (x)
        (cond
            ((vl-position x temp) t)
            ((setq temp (cons x temp)) nil)
        )
    )
    (vl-remove-if
       'foo
        lst
    )
)

And still greater performance is realized by separating the functions --

(defun RemoveDuplicatesAux ( x )
    (cond
        ((vl-position x index))
        ((null (setq index (cons x index))))
    )
)

(defun RemoveDuplicates ( lst / index )
    (vl-remove-if
       'RemoveDuplicatesAux
        lst
    )
)

[john.uhden]

@blogginsF6GT8 

Here are a couple of functions that work correctly, I think:

I named your list lst.

(defun remdupes1 (old / new)
  (foreach item old
    (if (not (vl-position item new))(setq new (cons item new)))
  )
  (reverse new)
)
;; But if your list is all numbers (real and/or integer),
;; then we should probably account for a fuzz factor:
(defun remdupes2 (old fuzz / new)
  (foreach item old
    (if (vl-every '(lambda (x)(not x))(mapcar '(lambda (x)(equal item x fuzz)) new))(setq new (cons item new)))
  )
  (reverse new)
)

Command: (remdupes1 lst)
(17.75 17.1875 18.3125 18.9375 18.125 18.5625 18.75 19.375 19.1875 19.8125 
19.625)
Command: (remdupes2 lst 1e-6)
(17.75 17.1875 18.3125 18.9375 18.125 18.5625 18.75 19.375 19.1875 19.8125 
19.625)

remdupes1 is fine for your list since each number is no more than 4 places.

But if you are using symbol values that are more precise (like the full value of pi), then you'd be better off using remdupes2.  Hmm, seems that remdupes2 work with strings as well, in fact probably a mixture of value types.

Command: (setq lst (list "a" "b" "c" "a" 2 pi 3.14159 4))
("a" "b" "c" "a" 2 3.14159 3.14159 4)

Command: (remdupes2 lst 1e-8)
("a" "b" "c" 2 3.14159 3.14159 4)

Command: (remdupes2 lst 1e-5)
("a" "b" "c" 2 3.14159 4)

[blogginsF6GT8]

Thank you so much guys for the great responses .. i have sooo much to learn. I'm going to take a bit of time and extract as much understanding as i can. 

once again, thanks to all 

[john.uhden]

@blogginsF6GT8 

We're here to help if you need it to unclog your understanding.

Just keep it in the same topic; don't start a new one (unless it's unrelated).