How to find manually installed packages?

Question

I installed a plain Ubuntu 10.10 on my computer and installed some software via apt-get. Last week I managed to break everything and just started from scratch, and I need to reinstall my software. Is there some way to create a list with all the packages that I have installed manually?

So that it would give me a list like texlive, ... and not texlive, texlive-dep1, textlive-dep2, ... and all the standard packages removed?

If I could somehow figure out which programs out of the regular install I have removed, that would be awesome too!

---

Update 2015-05-23: I now use Ansible as configuration management on my systems. There I specify the packages to install. This serves the goal, installing all the programs on a fresh system, even better. So this question might be the entry into some light configuration management.

Answer 1

With this suggestion, I'm assuming your old installation will still boot!

To replicate one set of packages on another machine:

On System A, run:

dpkg --get-selections | grep -v deinstall > my-selections

Move the my-selections file over to System B.

On System B, run:

dpkg --set-selections < my-selections

and then:

sudo apt-get dselect-upgrade

Important note: if you have installed packages from non-standard repositories and/or PPAs, you will also want to copy /etc/apt/sources.list and the contents of /etc/apt/sources.list.d/ from System A to System B before you run the upgrade.

You can use dpkg to see what you've removed as well (NB: this will also include packages that you manually installed and removed):

dpkg --get-selections | grep deinstall

You can see your results in the terminal, or, of course, redirect to a file.

Answer 2

This thread from superuser.com gives this solution:

aptitude search '?installed ?not(?automatic)'

Answer 3

If your apt logs are in /var/log/apt, something like this should work:

gunzip -c /var/log/apt/history.log.*.gz | grep "apt-get install"

Or if you want to get rid of some of the repetitive junk in the output:

gunzip -c /var/log/apt/history.log.*.gz | grep "apt-get install" \
  | cut -f4- -d" " | sort | uniq

Answer 4

You could use apt-mark, but I recommend debfoster:

sudo apt-get install debfoster
sudo debfoster

This will inspect all installed packages and figure out which ones are keeping the others installed:

texlive-full is keeping the following 161 packages installed:
  cm-super cm-super-minimal context doc-base dvipng feynmf
  fonts-gfs-artemisia fonts-gfs-baskerville fonts-gfs-bodoni-classic
  ...
Keep texlive-full? [Ynpsiuqx?], [H]elp:

As you answer "y" to each question (just push Enter to move quickly), debfoster will collect the package list and write them line-by-line to a file. By default this is at /var/lib/debfoster/keepers. It looks like this:

gnome-do
texlive-full
...

I configure debfoster via /etc/debfoster.conf to put this list at /etc/debfoster-keepers and track the file with etckeeper to keep history and backups. The answer here shows how to install a list of packages from a newline-delimited text file:

sudo apt-mark manual $(cat debfoster-keepers)

Note a limitation here, packages you purged have a '-' in front of them. So you want to remove those lines before calling apt-mark.

Even though the debfoster's website says that debfoster is deprecated in favor of aptitude, I prefer debfoster's prompt and simple configuration. It puts you in the middle of your package database and lets you clean things up, making the auto and manual packages more obvious.

Type "h" at the debfoster prompt to explain your options. Type "?" to see the package description. The how-to here might be useful.

Answer 5

comm -23 <(apt-mark showmanual | sort -u) <(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort -u)

Gives all manually installed packages (not system packages, not dependencies). For examples it shows build-essential but not gcc.

Answer 6

For those who prefer using apt-* tools, there's an apt-mark utility that allows both querying and modifying manual/auto state.

apt-mark showmanual

You can also specify pattern for packages you interested in:

apt-mark showmanual qemu\*

Answer 7

I'm able to pull everything by opening the log files in /var/log/apt/

I then go in and manually filter out the apt-get install packages. There may be a way to do this programatically but I'm not aware of it.

Answer 8

I finally got it now:

outfile="$(mktemp)"
pattern='(\[INSTALLIEREN\]|\[INSTALL\])'

if [[ -f "/var/log/aptitude.1.gz" ]]
then
        gunzip -c /var/log/aptitude.*.gz | grep -E "$pattern" | awk '{ print $2; }' > "$outfile"
fi

if [[ -f "/var/log/aptitude" ]]
then
        grep -E "$pattern" "/var/log/aptitude" | awk '{ print $2; }' >> "$outfile"
fi

sort "$outfile"
rm "$outfile"

Answer 9

cd /var/log/apt
cat  history.log | grep Commandline

this way you see the list of commands performed in the past.

Remove the grep code if you need more information:

less history.log

if the log is compressed (i.e. end with gz)

gunzip <filename>

to zip it again when finished you can do:

gzip <filename>

Answer 10

I didn't see any solutions here work for me, I have installed quite a few deb packages with dpkg and a few of the items I was particularly looking for were missing.

A rather lengthy one liner, but convenient to copy and paste would be:

export DPKG_INITIAL=$(mktemp) DPKG_INSTALLED=$(mktemp) DPKG_CUSTOM=$(mktemp) DPKG_DEPS=$(mktemp) zgrep -E '^Package' /var/log/installer/initial-status.gz | awk '{ print $2 }' > $DPKG_INITIAL ; awk '$3 !~ /install|remove|purge/ { next } { gsub(/remove|purge/, "uninstall", $3) ; gsub(/:.+/, "", $4) ; a[$4]=$3 } END { for (p in a) { if (a[p] == "install") { print p } } }' /var/log/dpkg.log | sort -u > $DPKG_INSTALLED ; comm -23 installed initial > $DPKG_CUSTOM ; function rdep() { apt-cache rdepends $1 | tail -n +3 | sed -e 's/^ //' -e '/^ /d' | cut -d':' -f1 | sort -u; } ; echo "$(for i in $(cat custom) ; do rdep $i ; done)" | sort -u > $DPKG_DEPS ; comm -23 custom deps > my-packages ; rm $DPKG_INITIAL $DPKG_INSTALLED $DPKG_CUSTOM $DPKG_DEPS

The above command saves a list of packages to a file in your current working directory named my-packages.

Explanation

I first built a list of packages that composed the baseline of packages selected during installation.

zgrep -E '^Package' /var/log/installer/initial-status.gz | awk '{ print $2 }' > initial

Followed by a long list of items installed in general.

awk '$3 !~ /install|remove|purge/ { next } { gsub(/remove|purge/, "uninstall", $3) ; gsub(/:.+/, "", $4) ; a[$4]=$3 } END { for (p in a) { if (a[p] == "install") { print p } } }' /var/log/dpkg.log | sort -u > installed

I then compared the two files initial and installed to list only the items unique to installed.

comm -23 installed initial > custom

From there I wanted to filter out the dependencies, this is where this method may miss some desired packages, it is unaware of dependencies that are also explicitly installed.

I wrote a quick bash function to shorten this steps in processing these items.

function rdep() { apt-cache rdepends $1 | tail -n +3 | sed -e 's/^ //' -e '/^ /d' | cut -d':' -f1 | sort -u; }

After that I passed each line from my file custom into this function with xargs.

echo "$(for i in $(cat custom) ; do rdep $i ; done)" | sort -u > deps

Once I had the long list of every possible dependency, (not sure on the every possible statement), I once again got the lines that were unique to a single file.

comm -23 custom deps > manual

And my finished list of packages is now in a file named manual available for me to review.

Answer 11

I think you want to use /var/log/apt/history.log and its friends to give the changes that have been made:

zcat /var/log/apt/history.log.*.gz | cat - /var/log/apt/history.log | grep -Po '^Commandline: apt(?:-get)? install (?!.*--reinstall)\K.*'

(Modified from source)

Will give you all the packages that have been installed at the command line using a variation of apt-get install.

It gets the zipped files in the old history.logs adds them with the current history.log and passes through grep, extracting lines with apt[-get] install [and/or reinstall] and showing the rest of the line (-o flag) which amounts to the package name(s).

This'll need a little modding (eg with sed) if you want just the package names on separate lines; an exercise for the reader!

It's worth noting that the synaptic package manager (gksu synaptic or kdesudo synaptic) in the "status" section has a "installed(manually)" list. If you mark the whole list for removal [don't apply it!!] you should be able to save the marked changes and get a list of packages that way. Caution: this locked up synaptic for me (calculating the dependency removals presumably).

Answer 12

Someone linked to https://unix.stackexchange.com/questions/3595/ubuntu-list-explicitly-installed-packages/3624#3624 which does have a good solution, but, it doesn’t behave correctly as the output from aptitude has changed. Here is an updated version, based around compared currently installed packages compared to 12.04 LTS. You will need aptitude installed, that is the only requirement.

aptitude search '~i !~M' -F '%p' | sort -u | tr -d ' ' > currentlyinstalled && wget -qO - http://mirror.pnl.gov/releases/precise/ubuntu-12.04.3-desktop-amd64.manifest | cut -f 1 | sort -u > defaultinstalled && comm -23 currentlyinstalled defaultinstalled

To break the above command down into parts, this bit outputs one package per line of everything installed on the system

aptitude search '~i !~M' -F '%p' | sort -u | tr -d ' ' > currentlyinstalled

And this downloads the default package list & crops the redundant information.

wget -qO - http://mirror.pnl.gov/releases/precise/ubuntu-12.04.3-desktop-amd64.manifest | cut -f 1 | sort -u > defaultinstalled

and comm compares the two files and outputs the packages which arent in the default list.

comm -23 currentlyinstalled defaultinstalled

Answer 13

#! /bin/sh
DEFAULT=`tempfile`
CURRENT=`tempfile`
cat /var/log/installer/initial-status.gz | gzip -d | grep '^Package:' | awk '{ print $2}' | sort -u > $DEFAULT
aptitude search '~i !~M' -F '%p' | sort -u | tr -d ' ' | awk '{ print $1}' > $CURRENT
comm -23 $CURRENT $DEFAULT
rm $DEFAULT
rm $CURRENT

Answer 14

Use a distribution manifest file as the base package set. Sort the manually installed results into package groups by architecture and section, so it's easier to focus on groups of packages (maybe you don't care about some sections).

https://gist.github.com/darrenleeweber/8cc570ff402f19af7fa4

#!/bin/bash

manifest_url='http://releases.ubuntu.com/releases/trusty/ubuntu-14.04.3-desktop-amd64.manifest'
manifest_file=$(echo $manifest_url | sed -e 's#.*/##g')
if [ ! -e $manifest_file ]; then
    wget -q $manifest_url
fi
cat $manifest_file | cut -f1 | sort -u > default_installed.txt

aptitude search '~i !~M' -F '%p' --disable-columns | sort -u > currently_installed.txt

comm -23 currently_installed.txt default_installed.txt > manually_installed.txt

# sort the 'mannually_installed.txt' packages by architecture and section
mkdir -p package_files
while read p; do
    apt-cache show $p > info.txt
    arch=$(grep -m1 'Architecture: ' info.txt | sed -e 's/Architecture: //')
    section=$(grep -m1 'Section: ' info.txt | sed -e 's/Section: //' -e 's/\//_/g')
    file="${arch}_${section}_packages.txt"
    echo $p >> "package_files/$file"
done <manually_installed.txt

rm info.txt

Answer 15

I didn't find one that was suitable for me, so I wrote a script. By analysis of the contents of /var/log/apt/history.log* and /var/log/aptitude, then sort by datetime, resulting in two lists, one is manually installed pkgs, another is installed but removed pkgs. Script with 'apt list' verification. a little deviation, because the use of dpkg installed and modified PPA source.

I put here, https://github.com/eexpress/eexp-bin/blob/master/self-installed-pkg.pl.

if anyone need test it, maybe need modify some code, for my log files are mixed in Chinese and English (specially RFC2822 datetime format), I need deal with it.