On 11.10:
$ vim.tiny --version
VIM - Vi IMproved 7.3 (2010 Aug 15, compiled Oct  6 2011 10:32:12)
Included patches: 1-154
Modified by pkg-vim-maintainers@lists.alioth.debian.org
Compiled by buildd@
Small version without GUI.  Features included (+) or not (-):
-arabic -autocmd -balloon_eval -browse +builtin_terms -byte_offset -cindent 
-clientserver -clipboard -cmdline_compl +cmdline_hist -cmdline_info -comments 
-conceal -cryptv -cscope -cursorbind -cursorshape -dialog -diff -digraphs -dnd 
-ebcdic -emacs_tags -eval -ex_extra -extra_search -farsi -file_in_path 
-find_in_path -float -folding -footer +fork() -gettext -hangul_input +iconv 
-insert_expand +jumplist -keymap -langmap -libcall -linebreak -lispindent 
-listcmds -localmap -lua -menu -mksession -modify_fname -mouse -mouse_dec 
-mouse_gpm -mouse_jsbterm -mouse_netterm -mouse_sysmouse -mouse_xterm 
+multi_byte -multi_lang -mzscheme -netbeans_intg -osfiletype -path_extra -perl 
-persistent_undo -printer -profile -python -python3 -quickfix -reltime 
-rightleft -ruby -scrollbind -signs -smartindent -sniff -startuptime 
-statusline -sun_workshop -syntax -tag_binary -tag_old_static -tag_any_white 
-tcl +terminfo -termresponse -textobjects -title -toolbar -user_commands 
-vertsplit -virtualedit +visual -visualextra -viminfo -vreplace +wildignore 
-wildmenu +windows +writebackup -X11 +xfontset -xim -xsmp -xterm_clipboard 
-xterm_save 
   system vimrc file: "$VIM/vimrc"
     user vimrc file: "$HOME/.vimrc"
      user exrc file: "$HOME/.exrc"
  fall-back for $VIM: "/usr/share/vim"
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H     -Wall -g -O2 -DTINY_VIMRC -D_FORTIFY_SOURCE=1      
Linking: gcc   -Wl,-Bsymbolic-functions -Wl,--as-needed -o vim    -lSM -lICE -lXpm -lXt -lX11 -lXdmcp -lSM -lICE -lm -ltinfo  -lselinux -ldl
There are only twelve features enabled (out of one hundred and twenty):
+builtin_terms      Some terminals supported
+cmdline_hist       Command line history
+fork()             Shell commands are forked
+iconv              Extra encoding conversions besides utf8<->latin1
+jumplist           jumplist history, 
                    So you can go back/forward with Ctrl-O/Ctrl-I
+multi_byte         Multibyte characters
+terminfo           Use terminfo instead of termcap
+visual             Visual mode supported (but no blockwise visual mode)
+wildignore         Allow wildcard patterns, to specify files to
                    ignore during filename completion
+windows            Support more than one buffer window
+writebackup        Backup files before overwriting (this either
                    provides this option, or defaults it to on)
+xfontset           X fontset support
The descriptions are based on the feature list link posted by @Caesium.
The missing keybindings etc may be because you are running vim in vi compatible mode - you can turn that off by doing :set nocompatible in vim or adding set nocompatible to you .vimrc file.
I did try Ctrl-W s and that did a split for me while running /usr/bin/vim.tiny while in compatible mode, so that might not be your problem. But I'm not sure what would be in that case. Maybe try making sure there is nothing in your .vimrc file that would stop that working.