dos2unix (将纯文本文件从DOS或Mac格式转换为Unix,反之亦然)

瑞兹 发表于 2021-01-14 09:10
浏览次数:
在手机上阅读

在类似Unix的操作系统上,dos2unix和unix2dos命令将纯文本文件从DOS或Mac格式转换为Unix,反之亦然。

查看英文版

目录

1 dos2unix 运行系统环境

2 dos2unix 描述

3 dos2unix 语法

4 dos2unix 例子

dos2unix 运行系统环境

Unix&Linux

dos2unix 描述

在DOS /Windows的文本文件,一个换行符,也被称为换行,是两个字符的组合:一个回车(CR)后跟一个换行(LF) 。在Unix文本文件中,换行符是单个字符:换行符(LF)。在Mac文本文件中,在macOS X之前,换行符是单个回车(CR)字符。如今,macOS使用Unix样式(LF)换行符。

除非强制转换,否则二进制文件将自动跳过。

非常规文件(例如目录和FIFO)将被自动跳过。

默认情况下,符号链接及其目标保持不变。可以选择替换符号链接,或者可以将输出写入符号链接目标。Windows不支持符号链接。Windows符号链接始终被替换,从而使目标保持不变。

Dos2unix是在SunOS / Solaris下模仿dos2unix建模的,并且具有类似的转换模式。

In DOS/Windows text files, a line break, also known as newline, is a combination of two characters: a Carriage Return (CR) followed by a Line Feed (LF). In Unix text files a line break is a single character: the Line Feed (LF). In Mac text files, prior to macOS X, a line break was single Carriage Return (CR) character. Nowadays macOS uses Unix style (LF) line breaks.

Binary files are automatically skipped, unless conversion is forced.

Non-regular files, such as directories and FIFOs, are automatically skipped.

Symbolic links and their targets are by default kept untouched. Symbolic links can optionally be replaced, or the output can be written to the symbolic link target. Symbolic links on Windows are not supported. Windows symbolic links are always replaced, keeping the targets unchanged.

Dos2unix was modelled after dos2unix under SunOS/Solaris and has similar conversion modes.

查看英文版

查看中文版

dos2unix 语法

dos2unix [options] [FILE ...] [-n INFILE OUTFILE ...]
unix2dos [options] [FILE ...] [-n INFILE OUTFILE ...]

选件

-

将后面的所有选项都视为文件名。例如,如果要转换名称以短划线开头的文件,请使用此选项。因此,要转换名为“ -foo ”的文件,可以使用以下命令:

dos2unix--foo或在新文件模式下:

dos2unix -n--foo out.txt

-ascii

仅转换换行符。这是默认的转换模式。

-iso

DOS和ISO -8859-1字符集之间的转换。另请参见“转换模式”部分。

-1252

使用Windows代码页1252(西欧)。

-437

使用DOS代码页437(美国)。这是用于ISO转换的默认代码页。

-850

使用DOS代码页850(西欧)

-860

使用DOS代码页860(葡萄牙语)。

-863

使用DOS代码页863(加拿大法语)。

-865

使用DOS代码页865(北欧)。

-7

将8位字符转换为7位空间。

-c,--convmode CONVMODE

设置转换模式。其中CONVMODE是以下之一:ascii7bitisomac,默认为ascii

-f,--force

强制转换二进制文件。

-h,--help

显示帮助并退出。

-k,--keepdate

使输出文件的日期戳与输入文件相同。

-L,--license

显示程序的许可证。

-l,--newline

添加其他换行符。

dos2unix:仅将DOS换行符更改为两个Unix换行符。在Mac模式下,只有Mac换行符更改为两个Unix换行符。

unix2dos:仅将Unix换行符更改为两个DOS换行符。在Mac模式下,Unix换行符更改为两个Mac换行符。

-m,--add-bom

在输出文件中写入UTF -8字节顺序标记。当输出编码不是UTF-8时,请勿使用此选项。另请参见UNICODE部分。

-n,--newfile INFILEOUTFILE...

新文件模式。转换文件INFILE并将输出写入文件OUTFILE。文件名必须成对给出,并且不应使用通配符名,否则您将丢失文件。

以新文件(配对)模式启动转换的人将是转换后文件的所有者。新文件的读/写权限将是原始文件的权限减去运行转换人员的umask。

-o,--oldfile FILE...

旧文件模式。转换文件FILE并覆盖输出。该程序默认在此模式下运行。可以使用通配符名称。

在旧文件(就地)模式下,转换后的文件与原始文件具有相同的owner,group和读取/写入 权限。同样,当文件被另一个对该文件具有写许可权的用户(例如,root用户)转换时。当无法保留原始值时,转换将中止。所有者的更改可能意味着原始所有者不再能够读取该文件。组的更改可能会带来安全风险,该文件对于不希望使用该文件的人来说可以使其可读。所有者,组和读/写权限的保留仅在Unix上受支持。

-q,-quiet

静音模式。禁止显示所有警告和消息。该返回值是零。除非使用了错误的命令行选项。

-s,--safe

跳过二进制文件(默认)。

-F,--follow-symlink

跟随符号链接并转换目标。

-R,--replace-symlink

将符号链接替换为转换后的文件(原始目标文件保持不变)。

-S,--skip-symlink

保持符号链接和目标不变(默认)。

-V,--version

显示版本信息并退出。

Mac模式

在正常模式下,将换行符从DOS转换为Unix,反之亦然。Mac换行符不会转换。

在Mac模式下,将换行符从Mac转换为Unix,反之亦然。DOS换行符不会更改。

要在Mac模式下运行,请使用命令行选项“ -c mac ”或使用命令“ mac2unix ”或“ unix2mac ”。

转换模式

ASCII

在“ ascii”模式下,仅转换换行符。这是默认的转换模式。

尽管此模式的名称是ASCII(这是7位标准),但实际模式是8位。转换Unicode UTF-8文件时,请始终使用此模式。

7bit

在这种模式下,所有8位非ASCII字符(值从128到255)都转换为7位空间。

iso

在Unix上,字符在DOS字符集(代码页)和ISO字符集ISO-8859-1(Latin-1)之间转换。没有等效ISO-8859-1的DOS字符(无法进行转换)将转换为点。没有DOS对应的ISO-8859-1字符的计数相同。

当仅使用选项“ -iso”时,dos2unix将尝试确定活动代码页。如果无法这样做,则dos2unix将使用默认代码页CP437,该代码页主要在美国使用。要强制使用特定的代码页,请使用选项“ -437”(美国),“-850”(西欧),“-860”(葡萄牙语),“-863”(法语加拿大)或“ -865”(北欧语) 。选项“ -1252”也支持Windows代码页CP1252(西欧)。对于其他代码页,请结合使用dos2unix和iconviconv可以在一长串字符编码之间进行转换。

切勿在Unicode文本文件上使用ISO转换。它将破坏UTF-8编码的文件。

一些例子:

从DOS默认代码页转换为Unix Latin-1:

dos2unix -iso -n in.txt out.txt

从DOS CP850转换为Unix Latin-1:

dos2unix -850 -n in.txt out.txt

从Windows CP1252转换为Unix Latin-1:

dos2unix -1252 -n in.txt out.txt

从Windows CP1252转换为Unix UTF-8(Unicode):

iconv -f CP1252 -t UTF-8 in.txt | dos2unix> out.txt

从Unix Latin-1转换为DOS默认代码页:

unix2dos -iso -n in.txt out.txt

从Unix Latin-1转换为DOS CP850:

unix2dos -850 -n in.txt out.txt

从Unix Latin-1转换为Windows CP1252:

unix2dos -1252 -n in.txt out.txt

从Unix UTF-8(Unicode)转换为Windows CP1252:

unix2dos  out.txt

注意:转换模式ascii7bitiso与SunOS / Solaris下的dos2unix / unix2dos相似。

统一码

编码方式

存在不同的Unicode编码。在Linux上,Unicode文件通常以UTF-8编码进行编码。在Windows上,Unicode文本文件可以UTF-8,UTF-16或UTF-16 big endian编码,但是大多数情况下以UTF-16格式编码。

转换次数

Unicode文本文件可以像常规文本文件一样具有DOS,Unix或Mac换行符。

dos2unixunix2dos的所有版本可以转换UTF-8编码的文件,因为UTF-8旨在与ASCII向后兼容。

支持Unicode UTF-16的dos2unix和unix2dos可以读取大小端的UTF-16编码文本文件。要查看dos2unix是否使用UTF-16支持类型构建,请输入“ dos2unix -V ”。

Windows版本的dos2unix和unix2dos始终将UTF-16编码的文件转换为UTF-8编码的文件。Unix版本的dos2unix / unix2dos设置为UTF-8时,会将UTF-16编码的文件转换为语言环境字符编码。使用locale命令找出什么是区域设置字符编码。

因为Windows和Unix都很好地支持UTF-8格式的文本文件,所以dos2unix和unix2dos没有写UTF-16文件的选项。所有UTF-16字符都可以用UTF-8编码。从UTF-16到UTF-8的转换不会丢失。当区域设置字符编码不是UTF-8时,在Unix上将跳过UTF-16文件,以防止意外丢失文本。当发生UTF-16到UTF-8转换错误时,例如,当UTF-16输入文件包含错误时,该文件将被跳过。

ISO和7位模式转换不适用于UTF-16文件。

字节顺序标记

在Windows上,Unicode文本文件通常具有字节顺序标记(BOM),因为默认情况下,许多Windows程序(包括记事本)都会添加BOM。另请参阅https://en.wikipedia.org/wiki/Byte_order_mark。

在Unix上,Unicode文件通常没有BOM。假定文本文件以语言环境字符编码进行编码。

如果文件具有BOM表,则dos2unix仅能检测文件是否为UTF-16格式。当UTF-16文件没有BOM表时,dos2unix会将其视为二进制文件。

dos2unixiconv结合使用可转换不带BOM的UTF-16文件。

Dos2unix绝不会在输出文件中写入BOM,除非您使用选项“ -m ”。

当输入文件具有BOM或使用选项“ -m ”时,Unix2dos将BOM写入输出文件。

例子

从Windows UTF-16(含BOM)转换为Unix UTF-8:

dos2unix -n in.txt out.txt

从Windows UTF-16(无BOM)转换为Unix UTF-8:

iconv -f UTF-16 -t UTF-8 in.txt | dos2unix> out.txt

使用BOM从Unix UTF-8转换为Windows UTF-8:

unix2dos -m -n in.txt out.txt

从Unix UTF-8转换为Windows UTF-16:

unix2dos  out.txt

递归转换

dos2unixfindxargs命令结合使用,以递归方式转换目录树结构中的文本文件。例如,要转换当前目录类型下目录树中的所有.txt文件:

find . -name * .txt | xargs dos2unix

本土化

使用环境变量 LANG选择主要语言。该LANG变量由几部分组成的。第一部分以小写字母表示语言代码。第二个是可选的,是用大写字母表示的国家/地区代码,后跟一个下划线。还有一个可选的第三部分:字符编码,后跟一个点。POSIX标准类型外壳的一些示例:

export LANG = nl

荷兰语

export LANG = nl_NL

荷兰语,荷兰

export LANG = nl_BE

比利时荷兰人

export LANG = es_ES

西班牙文,西班牙文

export LANG = es_MX

西班牙语,墨西哥

export LANG = en_US.iso88591

英文,美国,拉丁文1编码

export LANG = en_GB.UTF-8

英文,英国,UTF-8编码

在Unix系统上,可以使用命令区域设置来获取特定于区域设置的信息。

语言

使用LANGUAGE环境变量,您可以指定语言的优先级列表,以冒号分隔。Dos2unix优先于LANGUAGE而不是LANG。例如,首先是荷兰语,然后是德语:“ LANGUAGE = nl:de ”。您必须先通过将LANG(或LC_ALL)设置为“ C ”以外的值来启用本地化,然后才能通过LANGUAGE变量使用语言优先级列表。

如果您选择了不可用的语言,则会收到标准的英语消息。

DOS2UNIX_LOCALEDIR

使用环境变量DOS2UNIX_LOCALEDIR,可以推翻编译期间设置的LOCALEDIRLOCALEDIR用于查找语言文件。GNU的默认值为“ / usr / local / share / locale ”。选项--version显示使用的LOCALEDIR

示例(POSIX shell):

export DOS2UNIX_LOCALEDIR = $ HOME / share / locale

返回值

成功时,返回零。发生系统错误时,将返回上一个系统错误。对于其他错误,返回1。

在安静模式下,返回值始终为零,除非使用了错误的命令行选项。

dos2unix [options] [FILE ...] [-n INFILE OUTFILE ...]
unix2dos [options] [FILE ...] [-n INFILE OUTFILE ...]

Options

--

Treat all options that follow as file names. Use this option, for instance, if you want to convert files whose names start with a dash. So, to convert a file named "-foo", you can use this command:

dos2unix -- -foo Or in new file mode:

dos2unix -n -- -foo out.txt

-ascii

Convert only line breaks. This is the default conversion mode.

-iso

Conversion between DOS and ISO-8859-1 character set. See also section CONVERSION MODES.

-1252

Use Windows code page 1252 (Western European).

-437

Use DOS code page 437 (US). This is the default code page used for ISO conversion.

-850

Use DOS code page 850 (Western European)

-860

Use DOS code page 860 (Portuguese).

-863

Use DOS code page 863 (French Canadian).

-865

Use DOS code page 865 (Nordic).

-7

Convert 8 bit characters to 7 bit space.

-c, --convmode CONVMODE

Set conversion mode. Where CONVMODE is one of: ascii7bitiso, or mac, with ascii being the default.

-f, --force

Force conversion of binary files.

-h, --help

Display help and exit.

-k, --keepdate

Keep the date stamp of output file same as input file.

-L, --license

Display program's license.

-l, --newline

Add additional newline.

dos2unix: Only DOS line breaks are changed to two Unix line breaks. In Mac mode only Mac line breaks are changed to two Unix line breaks.

unix2dos: Only Unix line breaks are changed to two DOS line breaks. In Mac mode Unix line breaks are changed to two Mac line breaks.

-m, --add-bom

Write an UTF-8 Byte Order Mark in the output file. Never use this option when the output encoding is other than UTF-8. See also section UNICODE.

-n, --newfile INFILE OUTFILE ...

New file mode. Convert file INFILE and write output to file OUTFILE. File names must be given in pairs and wildcard names should not be used or you will lose your files.

The person who starts the conversion in new file (paired) mode will be the owner of the converted file. The read/write permissions of the new file will be the permissions of the original file minus the umask of the person who runs the conversion.

-o, --oldfile FILE ...

Old file mode. Convert file FILE and overwrite output to it. The program defaults to run in this mode. Wildcard names may be used.

In old file (in-place) mode the converted file gets the same ownergroup, and read/write permissions as the original file. Also, when the file is converted by another user who has write permissions on the file (e.g., user root). The conversion will be aborted when it is not possible to preserve the original values. Change of owner could mean that the original owner is not able to read the file any more. Change of group could be a security risk, the file could be made readable for persons for whom it is not intended. Preservation of owner, group, and read/write permissions is only supported on Unix.

-q, --quiet

Quiet mode. Suppress all warnings and messages. The return value is zero. Except when wrong command-line options are used.

-s, --safe

Skip binary files (default).

-F, --follow-symlink

Follow symbolic links and convert the targets.

-R, --replace-symlink

Replace symbolic links with converted files (original target files remain unchanged).

-S, --skip-symlink

Keep symbolic links and targets unchanged (default).

-V, --version

Display version information and exit.

Mac Mode

In normal mode line breaks are converted from DOS to Unix and vice versa. Mac line breaks are not converted.

In Mac mode line breaks are converted from Mac to Unix and vice versa. DOS line breaks are not changed.

To run in Mac mode use the command-line option "-c mac" or use the commands "mac2unix" or "unix2mac".

Conversion Modes

ascii

In mode "ascii" only line breaks are converted. This is the default conversion mode.

Although the name of this mode is ASCII, which is a 7 bit standard, the actual mode is 8 bit. Use always this mode when converting Unicode UTF-8 files.

7bit

In this mode all 8 bit non-ASCII characters (with values from 128 to 255) are converted to a 7 bit space.

iso

Characters are converted between a DOS character set (code page) and ISO character set ISO-8859-1 (Latin-1) on Unix. DOS characters without ISO-8859-1 equivalent, for which conversion is not possible, are converted to a dot. The same counts for ISO-8859-1 characters without DOS counterpart.

When only option "-iso" is used dos2unix will try to determine the active code page. When this is not possible dos2unix will use default code page CP437, which is mainly used in the USA. To force a specific code page use options "-437" (US), "-850" (Western European), "-860" (Portuguese), "-863" (French Canadian), or "-865" (Nordic). Windows code page CP1252 (Western European) is also supported with option "-1252". For other code pages use dos2unix in combination with iconviconv can convert between a long list of character encodings.

Never use ISO converion on Unicode text files. It will corrupt UTF-8 encoded files.

Some examples:

Convert from DOS default code page to Unix Latin-1:

dos2unix -iso -n in.txt out.txt

Convert from DOS CP850 to Unix Latin-1:

dos2unix -850 -n in.txt out.txt

Convert from Windows CP1252 to Unix Latin-1:

dos2unix -1252 -n in.txt out.txt

Convert from Windows CP1252 to Unix UTF-8 (Unicode):

iconv -f CP1252 -t UTF-8 in.txt | dos2unix > out.txt

Convert from Unix Latin-1 to DOS default code page:

unix2dos -iso -n in.txt out.txt

Convert from Unix Latin-1 to DOS CP850:

unix2dos -850 -n in.txt out.txt

Convert from Unix Latin-1 to Windows CP1252:

unix2dos -1252 -n in.txt out.txt

Convert from Unix UTF-8 (Unicode) to Windows CP1252:

unix2dos < in.txt | iconv -f UTF-8 -t CP1252 > out.txt

See also http://czyborra.com/charsets/codepages.html and http://czyborra.com/charsets/iso8859.html.

NOTE: Conversion modes ascii7bit, and iso are similar to those of dos2unix/unix2dos under SunOS/Solaris.

Unicode

Encodings

There exist different Unicode encodings. On Linux Unicode files are typically encoded in UTF-8 encoding. On Windows Unicode text files can be encoded in UTF-8, UTF-16, or UTF-16 big endian, but are mostly encoded in UTF-16 format.

Conversion

Unicode text files can have DOS, Unix or Mac line breaks, like regular text files.

All versions of dos2unix and unix2dos can convert UTF-8 encoded files, because UTF-8 was designed for backward compatibility with ASCII.

dos2unix and unix2dos with Unicode UTF-16 support can read little and big endian UTF-16 encoded text files. To see if dos2unix was built with UTF-16 support type "dos2unix -V".

The Windows versions of dos2unix and unix2dos convert UTF-16 encoded files always to UTF-8 encoded files. Unix versions of dos2unix/unix2dos convert UTF-16 encoded files to the locale character encoding when it is set to UTF-8. Use the locale command to find out what the locale character encoding is.

Because UTF-8 formatted text files are well supported on both Windows and Unix, dos2unix, and unix2dos have no option to write UTF-16 files. All UTF-16 characters can be encoded in UTF-8. Conversion from UTF-16 to UTF-8 is without loss. UTF-16 files will be skipped on Unix when the locale character encoding is not UTF-8, to prevent accidental loss of text. When an UTF-16 to UTF-8 conversion error occurs, for instance when the UTF-16 input file contains an error, the file will be skipped.

ISO and 7-bit mode conversion do not work on UTF-16 files.

Byte Order Mark

On Windows Unicode text files typically have a Byte Order Mark (BOM), because many Windows programs (including Notepad) add BOMs by default. See also https://en.wikipedia.org/wiki/Byte_order_mark.

On Unix Unicode files typically don't have a BOM. It is assumed that text files are encoded in the locale character encoding.

dos2unix can only detect if a file is in UTF-16 format if the file has a BOM. When an UTF-16 file doesn't have a BOM, dos2unix will see the file as a binary file.

Use dos2unix in combination with iconv to convert an UTF-16 file without BOM.

Dos2unix never writes a BOM in the output file, unless you use option "-m".

Unix2dos writes a BOM in the output file when the input file has a BOM, or when option "-m" is used.

Examples

Convert from Windows UTF-16 (with BOM) to Unix UTF-8:

dos2unix -n in.txt out.txt

Convert from Windows UTF-16 (without BOM) to Unix UTF-8:

iconv -f UTF-16 -t UTF-8 in.txt | dos2unix > out.txt

Convert from Unix UTF-8 to Windows UTF-8 with BOM:

unix2dos -m -n in.txt out.txt

Convert from Unix UTF-8 to Windows UTF-16:

unix2dos < in.txt | iconv -f UTF-8 -t UTF-16 > out.txt

Recursive Conversion

Use dos2unix in combination with the find and xargs commands to recursively convert text files in a directory tree structure. For instance to convert all .txt files in the directory tree under the current directory type:

find . -name *.txt | xargs dos2unix

Localization

LANG

The primary language is selected with the environment variable LANG. The LANG variable consists out of several parts. The first part is in small letters the language code. The second is optional and is the country code in capital letters, preceded with an underscore. There is also an optional third part: character encoding, preceded with a dot. A few examples for POSIX standard type shells:

export LANG=nl

Dutch

export LANG=nl_NL

Dutch, The Netherlands

export LANG=nl_BE

Dutch, Belgium

export LANG=es_ES

Spanish, Spain

export LANG=es_MX

Spanish, Mexico

export LANG=en_US.iso88591

English, USA, Latin-1 encoding

export LANG=en_GB.UTF-8

English, UK, UTF-8 encoding

For a complete list of language and country codes see the gettext manual: https://www.gnu.org/software/gettext/manual/gettext.html#Language-Codes

On Unix systems you can use to command locale to get locale-specific information.

LANGUAGE

With the LANGUAGE environment variable you can specify a priority list of languages, separated by colons. Dos2unix gives preference to LANGUAGE over LANG. For instance, first Dutch and then German: "LANGUAGE=nl:de". You have to first enable localization, by setting LANG (or LC_ALL) to a value other than "C", before you can use a language priority list through the LANGUAGE variable. See also the gettext manual: https://www.gnu.org/software/gettext/manual/gettext.html#The-LANGUAGE-variable

If you select a language that is not available you will get the standard English messages.

DOS2UNIX_LOCALEDIR

With the environment variable DOS2UNIX_LOCALEDIR the LOCALEDIR set during compilation can be overruled. LOCALEDIR is used to find the language files. The GNU default value is "/usr/local/share/locale". Option --version displays the LOCALEDIR that is used.

Example (POSIX shell):

export DOS2UNIX_LOCALEDIR=$HOME/share/locale

Return Values

On success, zero is returned. When a system error occurs the last system error will be returned. For other errors 1 is returned.

The return value is always zero in quiet mode, except when wrong command-line options are used.

查看英文版

查看中文版

dos2unix 例子

dos2unix

从stdin获取输入并将输出写入stdout。

dos2unix a.txt b.txt
dos2unix -o a.txt b.txt

以上两个命令都将执行相同的操作:在一个命令中转换并替换a.txt和b.txt。

dos2unix -k a.txt

转换并替换a.txt,同时保留原始日期戳。

dos2unix

Get input from stdin and write output to stdout.

dos2unix a.txt b.txt
dos2unix -o a.txt b.txt

Both of the above commands will do the same thing: convert, and replace, both a.txt and b.txt in one command.

dos2unix -k a.txt

Converts and replaces a.txt while keeping original date stamp.

查看英文版

查看中文版

其他命令行

date | du | dpost | dmesg | dirname | dircmp | dd | deluser | depmod | deroff | df | dhclient | diff | dig |

如此好文,分享给朋友
发表评论
验证码:
评论列表
共0条