I have a very big text file (10 GB). Both Notepad and Notepad++ tell me that the file is too big for them to open. What editor can handle such a large file? It's a log file.
-
1Most text editors put the entire files into RAM, so unless you have over 10GB of RAM, you're going to have a hard time finding a good text editor that does this. That said, I'm positive one exists.– Sasha ChedygovCommented Sep 3, 2009 at 5:46
-
21That's one heck of a text file– ColCommented Sep 3, 2009 at 7:28
-
6@NoCanDo - I would guess a log file of some kind, or a data file for processing.– RookCommented Sep 6, 2009 at 14:15
-
Definitely not kidding... And yes, it's a log file.– Lea CohenCommented Sep 6, 2009 at 19:20
-
1Is there any unuseful information in your file that can be parsed out of it to make your file smaller?– ralfordCommented Dec 13, 2009 at 23:48
|
Show 3 more comments
24 Answers
If you only need to read the file, I can suggest Large Text File Viewer.
-
2Sorry -1. This wasn't a very good editor. It froze up when opening a 7MB file. Commented Feb 10, 2011 at 19:26
-
19
-
@DJTripleThreat "This program was designed for viewing large (>1GB) text files." Source: swiftgear.com/ltfviewer/features.html– Connor WCommented May 22, 2011 at 11:28
-
7oh yeah? Notepad was designed to play 5.4TB HD videos as well. I can say that too. Doesn't mean it will do that though. Commented May 22, 2011 at 21:09
-
1I won't downvote this answer because
Large Text File Viewerdoes open very large files, BUT this is definitely not the best solution. Based on this answer I usedLarge Text File Viewerfor viewing large log files (excess of 2 Gb) and although it opened them, searching was very slow. I then triedV - The File Viewermentioned in an answer lower down and that was extremely fast... searches for text in a 2Gb file in a matter of seconds. V definitely gets my vote. Commented Oct 24, 2012 at 21:15
Stack Overflow ref Text editor to open big (giant, huge, large) text files.
Vim is a personal favorite.
- However, if you are working with timestamped entries in log file, it's easier to filter for a date range and look at smaller files.
- With other kinds of larger files, you could cut out specific line-number ranges (tools like sed and grep) to smaller files and analyze individually
- If it's some specific translation or searching to do, again sed, grep, etc. are good tools.
-
VIM is better than a regular text editor for big files, but it takes a while to open them anyway. I remember it taking like 30sec for a 200mb file (and looking like hanged during).– GnoupiCommented Sep 3, 2009 at 7:37
-
@Gnoupi, Yes I understand. The SO link had all the options and the rest of my answer described how I'd do it.– nikCommented Sep 3, 2009 at 8:03
-
@Gnoupi - VIM is an excellent editor, but it is a source code editor, and not specifically made for opening huge files. It can open very large files, but that's just because it is well designed. But for opening files of this size, I'd still rather go for something made for that purpose.– RookCommented Sep 6, 2009 at 22:02
-
It's unusual to see someone recommend sed on Windows. If anyone doesn't know, Unix-like tools for Windows are available several ways. CygWin and MSYS provide special command prompts. MinGW (which underlies MSYS) just provides the tools without a special command prompt, but doesn't necessarily provide everything you'd want. GNUWin32 just provides the specific tools you ask for.– user31438Commented Aug 6, 2012 at 19:00
UltraEdit is your best bet - I've opened files that were a few gigabytes in size with it. They also have some tips on opening large files.
It's not free, however.
Otherwise, you can use the Unix split utility from the command line - should be included in Cygwin or the Linux command line tools for Windows. You'll have much better luck dealing with the files if you've split them up into smaller chunks.
-
-
You might have to set the option to work without temporary files: Options - File Handling - Temporary Files gives you the relevant choices (check the second checkbox). Commented Sep 7, 2009 at 13:04
-
3I have used Ultraedit to open files that were nearly a hundred gigabytes large with no problems whatsoever. It works well.– BBlakeCommented Mar 30, 2010 at 12:18
-
Thanks. I had a big file I needed to work with and already have an ultraedit license.. Didnt think to try this after NotePad++, PSPad, vim failed me.– YablargoCommented Sep 27, 2013 at 17:58
The best one I tried was V - The File Viewer, when I needed to analyse a text file of more than 1 TB. It was by far the fastest. It's not free, though ($20).
-
If I could upvote this 10 times I would! I first tried
Large Text File Viewer(the accepted solution to this question) and it was disastrous... searching was SLOW.V - The File Vieweron the other hand is fantastic!! This answer definitely need more upvotes!! :) Commented Oct 24, 2012 at 21:19
EmEditor is a very nice editor all around, but also build with opening very large files in mind.
From their site, and I quote: " EmEditor Text Editor is a lightweight, yet extendable, easy-to-use text editor for Windows. EmEditor is very customizable, and it supports Unicode and powerful macros. EmEditor now supports very large files - up to 248 GB or 2.1 billion lines! EmEditor is certified for Windows Vista! The x64 Edition is also available! The portable version is also available for USB drive setup! "
-
This should be the answer. EmEditor has a large file controller which loads big files in background. I opened 10GB log file with this program within 60 seconds. Perfect!– useragedCommented Sep 22, 2011 at 12:48
-
1This was the only program that could reliably open my 32GB CSV files.– Robert HCommented Jan 8, 2013 at 18:37
-
This is great! I couldn't open just a 3.~ GB SQL dump with any of these other programs mentioned, but this one opened it immediately (probably because it didn't wait for it to load it all into memory first). Thanks~– JamesCommented Jul 29, 2013 at 17:14
Total Commander's lister can open very large files for reading. For editing you might also use some kind of Hex editor (like the free HxD), they can usually open and edit large files.
-
+1 for Lister. Immediatly opens any file, whatever it's size. Commented Apr 9, 2012 at 16:06
-
I used Lister successfully to search within a 10 Gig XML file.– MarcelCommented May 1, 2014 at 13:08
The Unix sed (stream editor) command can handle files of an arbitrary large (even infinite) file size, because it applies a script of editing commands to the stream of lines that pass through it. You can easily get it for Windows through the Cygwin distribution.
For instance, if you want to delete all files containing the string foo you write in the command prompt:
sed /foo/d <inputfile >outputfile
To substitute foo with bar write:
sed s/foo/bar/ <inputfile >outputfile
More complicated constructions are possible. Someone has even written a script to have sed solve the Towers of Hanoi puzzle.
-
You can also edit the file in place using
sed -i. To save a backup with the.bakextension usesed -ibakCommented Oct 5, 2009 at 9:26 -
The Cygwin version depends on cygwin libraries, so I would rather recommend the statically linked version from GnuWin (gnuwin32.sourceforge.net/packages/minised.htm) or from the UnixUtils package (unxutils.sourceforge.net)– ccpizzaCommented Nov 23, 2012 at 23:13
-
I've had luck with TextPad on files 2 GB in size with less ram than that, it takes a while to load up, but it works.
-
3It feature list implies that it does load the file into memory, even if it is virtual memory: "It can handle file sizes up to the largest contiguous chunk of 32-bit virtual memory."– dlamblinCommented Sep 6, 2009 at 21:34
-
textpad works brilliantly with large files. Plus - It's a great editor.– seanyboyCommented Sep 7, 2009 at 13:13
-
-
-
3It cannot handle files with larger size than avaible memory.– useragedCommented Sep 22, 2011 at 12:52
I have developed a Unicode editor called GigaEdit specifically to solve this issue of opening unlimited size files instantly by blocks of 1 MB with scrollbar to navigate effortlessly between blocks and a multi-threaded search feature that displays a result as they become available and allows you to go to matches instantly F3 or Shift+F3 even across blocks transparently.
WARNING: There is no UNDO after saving so make sure you know what you are doing.
For looking through large files (but only for read-only purposes), I use a Windows port of the UNIX less utility. It generally loads up any large file very quickly, and is pretty quick at searching for text patterns (once you get past its learning curve).
(Oh, and the other catch is that you have to run it within a command prompt window. It might not work if you were looking for something a bit more visually friendly.)
I don't recall where I found my current copy of it, but a quick web search shows a few possible places to download one, e.g.:
I would recommend VEDIT. It is about the only editor that is fast enough to realistically edit huge files. Its engine has been written in highly optimized assembly language.
No matter how big the file, VEDIT loads it in a fraction of a second (typically about 0.1 second). Most editors load the whole file in memory, which limits the file size and slows down opening the file. VEDIT only loads a part of the file at a time, and does not create a huge temporary file when opening the file (temporary files are created only when needed and as big as needed). In addition, VEDIT uses very little of computer memory. When I had a 300 MB file open, Task Manager reported 4.5 MB memory usage for VEDIT.
If you are just viewing the file and not editing, opening the file in read-only mode (with Fast Browse Mode enabled) makes VEDIT even faster. For example, you can scroll to any location in the file instantly, without any delay.
The standard (32-bit) version of VEDIT can handle files up to 2 GB. VEDIT Pro64 can handle files of any size.
To edit files larger than 2 GB on a 32-bit editor, you can split the file into smaller parts. You could use a separate tool for that, but VEDIT has built-in function to split and combine such files. For huge log files, splitting might be good idea anyway. You can then open all the parts for editing simultaneously and you can perform global searches among all of them, etc.
I had trouble loading a 5.5 GB binary file in a whole range of editors. Two I found that worked, and opened the file instantly:
HxD is my preferred one, since it is free, supports viewing and editing and is a stand alone. My large files are generally binary, so a hex editor is fine. But for viewing large text files, Altap Salamander has a better viewer.
I'll suggest gVim which can open such a huge file.
-
gVim is really good if your file fits in memory, still quite good if it fits in swap memory, but not so great if you have a really huge file and small memory, such as viewing Wikipedia XML dump files on a netbook. In that case an editor that can load just one big chunk of your file into memory at a time is better. Not sure which one is best though ... Commented Jan 11, 2013 at 8:24
You can also try Bare Tail
-
That seems to be just a GUI version of GNU CoreUtils'
tailcommand. gnuwin32.sourceforge.net/packages/coreutils.htm– dlamblinCommented Sep 6, 2009 at 21:30
I edited a file of 7 GB with PilotEdit. But it may take you an hour to open/save a 10 GB file. After you open the file, you can edit it quickly.
Under Windows, I use EditPad Pro from Just Great Software (there is a lite version too).
EditPad is the best editor I found. TextPad is also very good. Both now support UTF-8 file encoding.
I used to like pepper for this purpose. It's a little hard to find the abandoned windows port, and harder still to get it registered, though I managed about 2 years ago to contact Hekkelman about it. One of the features is that it does not load the whole file into memory.
-
That is a standard feature in this kind of editors. Opening from size till size, or fron line# till line#, right ?– RookCommented Sep 6, 2009 at 22:03
EmEditor is basically built for huge files. It's never balked at anything I'm thrown at it!
-
1If you want to advertise your product, I recommend you do so by contacting the team. You can buy advertisements to put on side, or even better, sponsor a tag (like text-editor). The point is, if you really want to advertise, that's the best way. Because if you keep doing it by poking all related questions and adding link to your product, you will be seen as spammer. And this rarely ends good. That said, have a good day, enjoy your stay around here, provide useful answers.– GnoupiCommented Jul 31, 2010 at 7:09
If you only want to view a section of a file, then you can use the following command in a DOS window.
type reallylargefile.txt | more
This will display file at a screen full at a time.
-
1
Assuming you only need to read the file, Microsoft's Windows 2003 resource kit includes list.exe.
Try TextPad. It's the best.
- Huge files can be edited, up to the limits of 32-bit virtual memory. See Specifications for the actual limits.
And then per the [Specifications]
- It can handle file sizes up to the largest contiguous chunk of 32-bit virtual memory.
- The number of concurrent edits is limited by available Windows resources.
-
Link and or explanation of how it supports big files is needed.– dlamblinCommented Sep 6, 2009 at 21:28
One approach is to break the giant file into smaller files. You can do that with Chainsaw. It takes a file of any size and outputs a bunch of smaller files from it.
Use FileSplitter to break the file into smaller files, and then use your favorite text editor!
Description: The File Splitter is a free Windows program that will split any file into pieces of any size and later combine the pieces to form the original file.
One very good editor for large files is Programmer's File Editor. I use it to ope files bigger than 15GB.