~/Uniq

Feb 3, 2022


The uniq command in Unix is used to filter out duplicate lines of sorted input.

flowchart TB
    subgraph Sorted
        A[Duplicate]
        B[Duplicate]
        C[Duplicate]
        D[Unique]
    end

    A --> B --> C --> D
    classDef duplicate fill:#ffcccc,stroke:#ff0000,stroke-width:2px,color:#000
    classDef unique fill:#ccffcc,stroke:#008000,stroke-width:2px,color:#000

    class A,B duplicate
    class D,C unique

flowchart TB
    subgraph Unsorted
        A[Duplicate]
        B[Duplicate]
        D[Unique]
        C[Duplicate]
    end

    A --> B --> D --> C
    classDef duplicate fill:#ffcccc,stroke:#ff0000,stroke-width:2px,color:#000
    classDef unique fill:#ccffcc,stroke:#008000,stroke-width:2px,color:#000

    class A duplicate
    class D,C,B unique

flowchart TB
    subgraph Sorted with -u
        A[Duplicate]
        B[Duplicate]
        C[Duplicate]
        D[Unique]
    end

    A --> B --> C --> D
    classDef duplicate fill:#ffcccc,stroke:#ff0000,stroke-width:2px,color:#000
    classDef unique fill:#ccffcc,stroke:#008000,stroke-width:2px,color:#000

    class A,B,C duplicate
    class D unique

Basics:

1
2
3
4
5

  uniq file.txt      # basic usage
  uniq -c file.txt   # count duplicates 
  uniq -d file.txt   # print duplicates 
  uniq -u file.txt   # print lines which are unique only
  uniq -i file.txt   # case insensitive

Scripts:

1
2

# Show unique connections to Nginx
awk '{print $1}' /var/log/nginx/access.log | sort | uniq

References

Tags: [linux] [cli]