1

I have a 4.6GB text file that I need to search through.

Surprisingly grep is fairly fast, but I need to be able to scroll around the file after searching for specific text.

Less would normally be my tool of choice but it is slow as molasses for this.

What pager and/or editor would work best on a 4.6GB file? My laptop has 16 GB of RAM total, so it needs to be somewhat efficient with its RAM usage.

user1175849
  • 111

2 Answers2

0

With a 1.5GB file vim works for me. A text search for non-existing string — i.e. through the whole file — took ≈10-15 sec. on my laptop. Note though, vim loads the file completely into the memory. Probably not a problem for you with 16GB RAM.

But I have this in my ~/.vimrc (unfortunately don't remember the original source):

" changes to open really big files
let g:LargeFile = 1024 * 1024 * 10
augroup LargeFile 
 autocmd BufReadPre * let f=getfsize(expand("<afile>")) | if f > g:LargeFile || f == -2 | call LargeFile() | endif
augroup END

function LargeFile()
 " no syntax highlighting etc
 set eventignore+=FileType
 " save memory when other file is viewed
 setlocal bufhidden=unload
 " is read-only (write with :w new_filename)
 setlocal buftype=nowrite
 " no undo possible
 setlocal undolevels=-1
 " display message
 autocmd VimEnter *  echo "The file is larger than " . (g:LargeFile / 1024 / 1024) . " MB, so some options are changed (see .vimrc for details)."
endfunction

You might also be interested in the similar question on another SE site, because there could be more tricks. E.g. I see there someone advises a vim plugin specifically for big files — never tried this one.

Hi-Angel
  • 4,810
0

Surprisingly grep is fairly fast, but I need to be able to scroll around the file after searching for specific text.

Maybe use grep options to show context around found entry?

Context control:
-B, --before-context=NUM  print NUM lines of leading context
-A, --after-context=NUM   print NUM lines of trailing context
-C, --context=NUM         print NUM lines of output context
Alex'LP
  • 91