I would like to convert (an) ANSI-encoded text file(s) (Win 1252) to UTF8 without BOM, ideally via a command-line call. My use-case: I export .tex files from Stata which I want to compile with LuaLaTeX. Stata apparently does not support UTF8, LuaLaTeX does not support anything but and thus chokes on some non-ASCII characters. From inside Stata I can call shell commands, so it would be nice if I could do the conversion on the fly from within my Stata scripts.
So ideally I would like to be able to call a command like e.g. convert2UTF.cmd file.tex
. Another good option would be some batch conversion of files within a folder (e.g. convert all files with *stata.tex). In addition it would be great if the solution would work with default Windows tools (minimum Win 7, even better XP).
Similar questions have been asked here before. The Cygwin/GnuWin32 approach is problematic since I would like to be able to convert without having to install extra software on a machine. The powershell approach looks promising, but apparently out-file -en utf8
saves the file with BOM.
Another powershell approach that seems to convert to UTF8 without BOM is
foreach($i in ls -recurse -filter "*.*") {
if (
$i.Extension.ToLower() -eq ".tex"
) {
$MyFile = Get-Content $i.fullname
[System.IO.File]::WriteAllLines($i.fullname, $MyFile)
}
}
Unfortunately I cannot figure out how to run it. I saved it as a powershell script into the same folder as the .tex files, but when I run it, it does not touch them. So apparently there is something missing. Needless to say that my powershell knowledge is close to nothing. Also, I would like to pass a filename as an argument, when calling it from Stata.
[System.IO.File]::WriteAllLines(out.tex, $MyFile)
to the powershell, I get a ParserError...WriteAllLines
or passing them in the wrong order.$MyFile = Get-Content in.tex
followed by[System.IO.File]::WriteAllLines(out.tex, $MyFile)
and I getCategoryInfo: ParserError: (CloseParenToken:TokenId) [], ParentContainsErrorRecordExceptionFullyQualifiedErrorId : MissingEndParenthesisInMethodCall