Why Command Substitiution Works And Pipeline Not

Question

I am practing | pipeline to pass output.

Firstly, I tried basename as

$ basename -a $(ls test_directory/*) 

This line works as expected and showed only file's name without directory's.

But when I try to pipe the output of ls to basename

$ ls test_directory/* | basename -a

Error message is here:

basename: missing operand
Try 'basename --help' for more information.

I found Pass the output of previous command to next as an argument on Unix & Linux on which one of the answer mentioned:

1. Passing input by stdin:

   ls | wc -l 
   

   This will count the lines in the output of ls

2. Passing input by command line arguments:

   wc -l $(ls)
   

   This will count lines in the list of files printed by ls

I can't understand the difference between "the output of ls" and "files printed by ls". Aren't they same thing? I want to understand the difference.

And I want to understand why ls td/* | basename -a didn't work.

Answer 1

To explain the difference between ls | wc -l and wc -l $(ls). I am going to take an example.

1. ls | wc -l

As mentioned in the post that this command will count the lines in the output of `ls`.

For instance I have a directory ~/Desktop/Practice/python which contains 4 python scripts.
Now if I use the above command, I get the following output:

4

This means the wc -l counts the output of ls which returned 4 python scripts hence count became 4

2. wc -l $(ls)

Again as suggested by the post, this command will count lines in the list of files printed by `ls`.
Since in this case you are passing `ls` as an argument to the command

Now if I use the above command in the same directory I get following output:

12 palindrome1.py  
11 palindrome2.py
12 palindrome3.py
6 palindrome4.py
41 total

This shows that wc -l this time counted the number of lines present in each individual python script which was returned to it by ls.

Difference b/w command line argument and standard input

The actual difference lies in the source of a command. A command could take input using either "commnad line arguments" or by taking a "Standard Input". Hence it is up to the developer of the command how to take input for a particular command.

The pipe passes the standard output of one command to the second command as standard input. Hence if the second command cannot process standard input it will throw an error. However some commands take both command line arguments and standard input like "wc" but the result is different because wc processes standard input and command line arguments differently.

In your case it was basename. If you read the man page of basename :

NAME
       basename - strip directory and suffix from filenames
SYNOPSIS
       basename NAME [SUFFIX]
       basename OPTION... NAME...
DESCRIPTION
       Print NAME with any leading directory components removed.  If specified, also remove a trailing SUFFIX.
   Mandatory arguments to long options are mandatory for short options too.

   -a, --multiple
          support multiple arguments and treat each as a NAME

It is clearly specified that basename takes command line arguments for it's input. Hence pipe will not work.

How to make pipe work

As suggested by @bac0n, to execute the basename with pipe you need to use xargs.

ls | xargs basename -a

This works because xargs takes standard input and passes to another command as "command line argument".

Answer 2

The easiest way that I can demonstrate it:

Some commands are built to accept stdin. Take cat as an example. You can pipe the output of previews command to cat:

$ echo hi | cat

Some commands only get arguments as input, for example echo. If you run:

$ ls | echo

It would pipe the output of ls to echo. But echo won't receive it. To use echo you have to give it some arguments:

$ echo "I'm an argument"

If you need to use a command's output as an argument to some other command (For example in a case when it only accepts arguments and not stdin), you have different choices. One is to use command substitution:

$ echo $(ls)

In this case ls would run then the output of ls would be passed as arguments to the echo.

Same for basename. You have to pass arguments.

Also there are commands which are able to take argument and stdin at the same time. For example cat. If you pipe something to cat it would print it out. However for an argument it except you to pass a filename to prints its contents out.