Bash

bash history

Remove and prevent duplicates in bash history

Step 1 - remove existing duplicates in .bash_history and create a new temporary file named unduped_history.

# change to the home directory
cd ~/

# remove duplicates and create unduped_history
nl ~/.bash_history | sort -k 2  -k 1,1nr| uniq -f 1 | sort -n | cut -f 2 > unduped_history

Step 2 - copy the unduped_history and overwrite .bash_history.

cp unduped_history ~/.bash_history

# unduped_history can be removed now
rm unduped_history 

• For zsh history, just replace .bash_history with .zsh_history

Step 3 - to prevent duplicate history entries going forward, add this line to the .bashrc or .zshrc if applicable.

export HISTCONTROL=ignoreboth:erasedups

• Linux