php: check if an array has duplicates

Question

I'm sure this is an extremely obvious question, and that there's a function that does exactly this, but I can't seem to find it. In PHP, I'd like to know if my array has duplicates in it, as efficiently as possible. I don't want to remove them like array_unique does, and I don't particularly want to run array_unique and compare it to the original array to see if they're the same, as this seems very inefficient. As far as performance is concerned, the "expected condition" is that the array has no duplicates.

I'd just like to be able to do something like

if (no_dupes($array))
    // this deals with arrays without duplicates
else
    // this deals with arrays with duplicates

Is there any obvious function I'm not thinking of?
How to detect duplicate values in PHP array?
has the right title, and is a very similar question, however if you actually read the question, he's looking for array_count_values.

Answer 1

I know you are not after array_unique(). However, you will not find a magical obvious function nor will writing one be faster than making use of the native functions.

I propose:

function array_has_dupes($array) {
   // streamline per @Felix
   return count($array) !== count(array_unique($array));
}

Adjust the second parameter of array_unique() to meet your comparison needs.

Answer 2

Performance-Optimized Solution

If you care about performance and micro-optimizations, check this one-liner:

function no_dupes(array $input_array) {
    return count($input_array) === count(array_flip($input_array));
}

Description:
Function compares number of array elements in $input_array with array_flip'ed elements. Values become keys and guess what - keys must be unique in associative arrays so not unique values are lost and final number of elements is lower than original.

Warning:
As noted in the manual, array keys can be only type of int or string so this is what you must have in original array values to compare, otherwise PHP will start casting with unexpected results. See https://3v4l.org/7bRXI for an example of this fringe-case failure mode.

Proof for an array with 10 million records:

• The top-voted solution by Jason McCreary: 14.187316179276s 🐌🐌🐌🐌🐌🐌🐌🐌🐌🐌🐌🐌🐌🐌
• The accepted solution by Mike Sherov: 2.0736091136932s 🐌🐌
• This answer's solution: 0.14155888557434s 🐌/10

Test case:

<?php

$elements = array_merge(range(1,10000000),[1]);

$time = microtime(true);
accepted_solution($elements);
echo 'Accepted solution: ', (microtime(true) - $time), 's', PHP_EOL;

$time = microtime(true);
most_voted_solution($elements);
echo 'Most voted solution: ', (microtime(true) - $time), 's', PHP_EOL;

$time = microtime(true);
this_answer_solution($elements);
echo 'This answer solution: ', (microtime(true) - $time), 's', PHP_EOL;

function accepted_solution($array){
 $dupe_array = array();
 foreach($array as $val){
  // sorry, but I had to add below line to remove millions of notices
  if(!isset($dupe_array[$val])){$dupe_array[$val]=0;}
  if(++$dupe_array[$val] > 1){
   return true;
  }
 }
 return false;
}

function most_voted_solution($array) {
   return count($array) !== count(array_unique($array));
}

function this_answer_solution(array $input_array) {
    return count($input_array) === count(array_flip($input_array));
}

Notice that accepted solution might be faster in certain condition when not unique values are near the beginning of huge array.

Answer 3

You can do:

function has_dupes($array) {
    $dupe_array = array();
    foreach ($array as $val) {
        if (++$dupe_array[$val] > 1) {
            return true;
        }
    }
    return false;
}

Answer 4

$hasDuplicates = count($array) > count(array_unique($array)); 

Will be true if duplicates, or false if no duplicates.

Answer 5

$duplicate = false;

 if(count(array) != count(array_unique(array))){
   $duplicate = true;
}

Answer 6

Here's my take on this… after some benchmarking, I found this to be the fastest method for this.

function has_duplicates( $array ) {
    return count( array_keys( array_flip( $array ) ) ) !== count( $array );
}

…or depending on circumstances this could be marginally faster.

function has_duplicates( $array ) {
    $array = array_count_values( $array );
    rsort( $array );
    return $array[0] > 1;
}

Answer 7

To remove all the empty values from the comparison you can add array_diff()

if (count(array_unique(array_diff($array,array("")))) < count(array_diff($array,array(""))))

Reference taken from @AndreKR answer from here

Answer 8

Two ways to do it efficiently that I can think of:

1. inserting all the values into some sort of hashtable and checking whether the value you're inserting is already in it(expected O(n) time and O(n) space)

2. sorting the array and then checking whether adjacent cells are equal( O(nlogn) time and O(1) or O(n) space depending on the sorting algorithm)

stormdrain's solution would probably be O(n^2), as would any solution which involves scanning the array for each element searching for a duplicate

Answer 9

Find this useful solution

function get_duplicates( $array ) {
    return array_unique( array_diff_assoc( $array, array_unique( $array ) ) );
}

After that count result if greater than 0 than duplicates else unique.

Answer 10

Keep it simple, silly! ;)

Simple OR logic...

function checkDuplicatesInArray($array){
    $duplicates=FALSE;
    foreach($array as $k=>$i){
        if(!isset($value_{$i})){
            $value_{$i}=TRUE;
        }
        else{
            $duplicates|=TRUE;          
        }
    }
    return ($duplicates);
}

Regards!

Answer 11

I'm using this:

if(count($array)==count(array_count_values($array))){
    echo("all values are unique");
}else{
    echo("there's dupe values");
}

I don't know if it's the fastest but works pretty good so far

Answer 12

One more solution from me, this is related to performance improvement

$array_count_values = array_count_values($array);
if(is_array($array_count_values) && count($array_count_values)>0)
{
   foreach ($array_count_values as $key => $value)
   {
      if($value>1)
      {
        // duplicate values found here, write code to handle duplicate values            
      }
   }
}

Answer 13

The shortest one

function no_dupes($array) {
    return $array === array_unique($array);
}

Answer 14

As you specifically said you didn't want to use array_unique I'm going to ignore the other answers despite the fact they're probably better.

Why don't you use array_count_values() and then check if the resulting array has any value greater than 1?

Answer 15

Php has an function to count the occurrences in the array http://www.php.net/manual/en/function.array-count-values.php

Answer 16

You can do it like that way also: This will return true if unique else return false.

$nofollow = (count($modelIdArr) !== count(array_unique($modelIdArr))) ? true : false;

Answer 17

The simple solution but quite faster.

$elements = array_merge(range(1,10000000),[1]);

function unique_val_inArray($arr) {
    $count = count($arr);
    foreach ($arr as $i_1 => $value) {
        for($i_2 = $i_1 + 1; $i_2 < $count; $i_2++) {
            if($arr[$i_2] === $arr[$i_1]){
                return false;
            }
        }
    }
    return true;
}

$time = microtime(true);
unique_val_inArray($elements);
echo 'This solution: ', (microtime(true) - $time), 's', PHP_EOL;

Speed - [0.71]!

Answer 18

function hasDuplicate($array){
  $d = array();
  foreach($array as $elements) {
    if(!isset($d[$elements])){
      $d[$elements] = 1;
    }else{
      return true;
    } 
  } 
  return false;
}