NAME nmrPipe - Vector Processing of Spectral Data SYNOPSIS nmrPipe [-fn fnName] < inFile > outFile [-in inName [-nofs]] [-out outName [-ov] [-nofs] [-inPlace]] [-tty] [-fdata loc1 parm1...] [-xi | -dt | -di] [-ai | -ad | -ac] [-select y1 yn [z1 zn...] [-inside | -outside]] [-verb [ver- boseLevel]] [-inc verboseIncrement] [-dbg debugReportLevel] [-X11] [-noSig] [-help [-allHelp]] [-client serverListName] [-server portFileName] [-sID serverID] [-inSock inSocket- Port] [-outSock outSocketPort] [-path pathName] [-smt ser- verMaxTries] [-sto serveruSecTimeOut] [-buff ioBuffCount] [-ibuff inputBuffCount] [-obuff outputBuffCount] [-mt max- Tries] [-to uSecTimeOut] [-noRd] [-noWr] [-inSwap] [-outSwap] AUTHORS The NMRPipe System of spectral analysis tools was designed and implemented by Frank Delaglio at the National Institutes of Health, Laboratory of Chemical Physics, NIDDK, Bethesda MD, in the Section on Biophysical NMR directed by Ad Bax. The software includes contributions from Guang Zhu, Geerten Vuister, Lewis Kay, John Pfeifer, Dan Garrett, Dennis Benja- min, Toshio Yamazaki, Dean Carlson, E. Neil Lewis, and Stephan Grzesiek. The software can be cited by referring to the following publication: NMRPipe: a multidimensional spectral processing system based on UNIX Pipes F. Delaglio, S. Grzesiek, G. Vuister, G. Zhu, J. Pfeifer, and A. Bax J. Biomol. NMR, 6 (1995) 277-293. Comments on the software are welcome, and questions will be answered when possible. Electronic correspondence can be addressed to delaglio@speck.niddk.nih.gov. DESCRIPTION The nmrPipe program is the central part of a system of tools for multi-dimensional spectral processing using UNIX pipes. It applies a given spectral processing function, for instance a one-dimensional Fourier Transform, to a series of spectral data vectors; successive applications of the nmrPipe program are used to build up a processing scheme for a single dimension of a multidimensional data set. When used in conjunction with data input and output programs such as xyz2pipe and pipe2xyz, nmrPipe can be used to construct complete multi-dimensional processing schemes. In most cases, nmrPipe processing schemes are constructed as UNIX shell-scripts, which are text files used to execute nmrPipe and related programs. In practice, these shell-scripts just define the input data source, the list of processing functions to apply, and the output data destina- tion. The shell scripts can be constructed with a UNIX text-editor such as vi, or inside a graphical interface such as nmrDraw. Sample shell-scripts for various purposes are given in the EXAMPLES section below, as well as in the manual pages of related commands. The following steps are involved in a typical processing session: Convert the spectrometer format data to nmrPipe for- mat; this is usually performed by editing a shell-script which runs a conversion program such as bruk2pipe or var2pipe. The conversion script defines the input and output data names, and the acquisition parameters. Inspect the raw data, and choose parameters such as phasing; this step will commonly be performed in the interactive graphical interface nmrDraw. Create and execute a processing scheme; this step involves editing a shell-script which runs several copies of nmrPipe, each defined to perform one aspect of the overall processing. Inspect the processed result, to decide if any addi- tional processing is required, for instance re-phasing or baseline correction. As before, this step will commonly be performed in the interactive graphical interface nmrDraw. BACKGROUND: UNIX FILTERS AND PIPES Many of the NMRPipe System's advantages derive from its use of specific features in the UNIX operating system, so some details of UNIX are reviewed here. The nmrPipe program operates like a UNIX filter; this means that it reads an input stream of data, processes the data in some way, and writes the result as an output stream. In UNIX terminology, the input stream of data is called the standard input, and the output stream of data is called the standard output. By default, standard input is data entered at the keyboard, but commands can be constructed so that the input is taken from a file instead; this is called input redirection. On the other hand, data written to the standard output is displayed on the computer screen, but commands can also be constructed so that the output is written to a file instead; this is called output redirection. Input redirection is performed by the command character "<", while output redirection is performed by the command character ">". Programs that read standard input and write results to stan- dard output can also be connected together by UNIX pipes. Pipes allow separate UNIX commands to be connected together in a series, where the output of one program is automati- cally used as the input of the next program. A series of programs connected this way is sometimes called a pipeline. A pipe is specified in a UNIX command line by the "|" verti- cal bar character. USING NMRPIPE The nmrPipe program operates on data that has been converted from the spectrometer format by utilities such as var2pipe or bruk2pipe. When using the nmrPipe program to process converted spectral data, the following information is speci- fied: 1. Source of the input data to process. 2. Processing function to apply. 3. Parameters required for the processing function. 4. Destination of the processed output data. If this information is not specified explicitly in the com- mand line, default values will be used, as outlined below. For simplicity, some of the following examples are incom- plete commands (e.g. missing input or output); they are indicated as "command fragments" in the text. SOURCE OF INPUT DATA By default, the nmrPipe program reads its standard input for data to process. This means that UNIX facilities can also be used to re-direct the program's input so that it comes either from a file or from another program which writes to standard output. In addition, the argument -in inName can also be used to specify that input should be read from a given file. The following command fragments all show nmrPipe using data from the 2D spectral data file test.fid as input (note that the UNIX command cat(1v) displays the contents of a file): nmrPipe < test.fid cat test.fid | nmrPipe nmrPipe -in test.fid SPECIFYING THE PROCESSING FUNCTION By default, the nmrPipe program applies the NULL processing function, which leaves the data unchanged. A different pro- cessing function can be selected by using the -fn fnName argument. The names of all available nmrPipe functions can be listed by using the -help flag. Most nmrPipe processing functions perform the same operation on all 1D vectors from the input stream. For example, the function named FT is used to apply a Com- plex Fourier Transform to all the 1D vectors from the input data. So, the command fragment for transforming all the 1D vectors in the acquisition dimension of the 2D spectral data file test.fid might look like this: nmrPipe -in test.fid -fn FT Note that each instance of nmrPipe can have only one func- tion as specified by the -fn fnName option. SPECIFYING PROCESSING PARAMETERS Depending on the processing function selected, nmrPipe accepts other arguments which define relevant processing parameters. The arguments associated with a given function can be displayed by using the -fn fnName argument along with the -help flag. This will generate a brief description of the specified function, its arguments, and their default values. For instance, the following example shows the com- mand used to display the arguments for the adjustable sine window function, SP: nmrPipe -fn SP -help This command will produce the following help text: SP: Adjustable Sine Window. [SINE] -off offset [0.0] Sine Start*PI. -end end [1.0] Sine End*PI. -pow exp [1.0] Sine Exponent. -size aSize [APOD] Apodize Length. -start aStart [1] Apodize Start. -c fScale [1] Point 1 Scale. -one Outside = 1. -inv Invert Window. Each line shows a command flag followed by the name of its argument value if any. The default parameters for each argu- ment are shown in square braces "[]"; a terse description of the argument is given on the right. Floating-point and integer parameters can be distinguished by their default values. In addition to this help text, many processing functions also have a more extensive manual page, which can be viewed by using the man command, for example: man SP In the help listing above, the default parameters call for a sinusoid with a starting point (offset) of 0.0*Pi radians, and an endpoint of 1.0*Pi radians. This means that when the function SP is used with no additional arguments, it will generate an exact sine-bell window, as in this command frag- ment: nmrPipe -in test.fid -fn SP The function can be altered by including other arguments; for example, this command fragment: nmrPipe -in test.fid -fn SP -off 0.5 -pow 2 -c 0.5 will generate a cosine-squared window which also multiplies the first point in every 1D vector by 0.5. In the help listing for SP above, the default length of the window function is listed as "APOD", which means that the default value will be taken from the time-domain size recorded in the data during conversion. Default values for many functions may also be expressed in terms of "SIZE" or "XSIZE" (the 1D size of the current dimension), or "YSIZE" (the 1D size of the next dimension). Note that these upper-case keywords should not be used in a command line; they are just reminders to the user that the default values depend on the parameters stored in the given data file. DESTINATION OF OUTPUT DATA By default, the nmrPipe program writes its processed data to standard output. As before, this means that UNIX facilities can be used to re-direct the program's output so that it goes either to a file or to another program which reads from standard input. In addition, the argument -out outName can also be used to specify that output should be written to a given file. The following command fragments all show nmrPipe writing data to the output file test.dat (note that the UNIX command dd is a data copying program): nmrPipe -in test.fid > test.dat nmrPipe -in test.fid | dd of=test.dat nmrPipe -in test.fid -out test.dat CREATING PROCESSING SCHEMES A complete instance of the nmrPipe program includes a data source, processing function specification, and a data desti- nation, as outlined above. So, for example, a complete com- mand to Fourier Transform the acquisition vectors of test.fid, with the result saved in test.ft, might look like this: nmrPipe -in test.fid -fn FT -out test.ft As mentioned before, the nmrPipe program will apply only a single processing function; however, in most spectral pro- cessing schemes, several functions must be applied. To achieve this, multiple instances of the nmrPipe program can be joined together by UNIX pipes. For example, the following scheme applies both a window function and a Fourier Transform to the acquisition vectors in file test.fid: nmrPipe -in test.fid -fn SP \ | nmrPipe -fn FT -out test.ft Notice that the above command includes two instances of the nmrPipe program, connected by a pipe. The character "\" is used at the end of the first line to indicate that the com- mand is continued on the next line. Note well that there should be no trailing blank spaces after the "\" character, or the processing scheme will not work. Following the example above, additional processing functions can be included simply by adding instances of nmrPipe. Here, the zero filling function ZF is added to the scheme above: nmrPipe -in test.fid -fn SP \ | nmrPipe -fn ZF \ | nmrPipe -out test.ft -fn FT The scheme above performs a window function, zero fill, and Fourier Transform on all vectors in the acquisition dimen- sion of file test.fid, and saves the processed result in test.ft. Considered in more detail, the scheme above con- sists of three instances of nmrPipe, connected by pipes, and running at the same time. The first instance of nmrPipe reads some data from test.fid, applies the window function SP, and writes the result vectors to the pipeline. At the same time, the second instance of nmrPipe reads these result vectors from the pipeline, applies the zero fill function ZF, and writes the zero-filled vectors to the pipeline. The third instance of nmrPipe reads zero-filled vectors from the pipeline, applies the Fourier Transform function FT, and writes the result vectors to the file test.ft. It is worthy to emphasize that a stream of vectors is moving continuously through the pipeline, so that all three functions are active and running asynchronously during processing. PROCESSING SCHEMES WITH TRANSPOSE FUNCTIONS The examples so far only process vectors from the acquisi- tion dimension of the input data. In order to process the next dimension, the transposition function TP can be included in the pipeline; it exchanges the row and column vectors in the 2D input planes, accommodating real, complex and hypercomplex data. So, for example, the following scheme is a simple 2D Fourier Transform of the 2D hypercom- plex (e.g. States Mode) file test.fid: nmrPipe -in test.fid -fn FT -di \ | nmrPipe -fn TP \ | nmrPipe -out test.ft -fn FT -di In this 2D example, the first instance of nmrPipe reads the input data from test.fid, applies a Fourier Transform to the input vectors, deletes the imaginary data for this dimension as requested by the -di flag, and writes processed results to the pipeline. The second instance of nmrPipe collects all the vectors in the 2D plane, and only then begins to write vectors in transposed order to the pipeline. The third instance of nmrPipe reads the transposed vectors from the pipeline, applies a Fourier Transform, deletes the ima- ginaries, and writes the result to file test.ft. This means that the second Fourier Transform in the example above is idle while the first Fourier Transform is running, since it will not receive data vectors until the transpose step is finished. Processing schemes can include more than one transpose step. In the following example, the automated baseline correction function POLY -auto is applied to the acquisition dimension only after the 2D data is Fourier Transformed in both dimen- sions: nmrPipe -in test.fid \ | nmrPipe -fn SP -off 0.5 \ | nmrPipe -fn FT -di \ | nmrPipe -fn TP \ | nmrPipe -fn SP -off 0.5 \ | nmrPipe -fn FT \ | nmrPipe -fn PS -p0 90.0 -di \ | nmrPipe -fn TP \ | nmrPipe -fn POLY -auto \ | nmrPipe -out test.ft PROCESSING SCHEMES AS SHELL-SCRIPTS Multi-line processing schemes like the ones above will usu- ally be typed into a text file first, and then executed as a UNIX shell script. Note that some versions of UNIX may require that the shell-script is identified as a C-shell script in order to work properly. This means that the very first line of every nmrPipe shell script should be: #!/bin/csh As a reminder of this, the schemes in the EXAMPLES section below begin with "#!/bin/csh". Schemes in shell-script files easy to modify and re-use as they are developed. For convenience, the -fn fnName option is often omitted at the head or tail of a processing pipe- line, so that the processing functions listed in the inte- rior of the pipeline can be changed or adjusted without effecting the input or output specification. For example, this processing scheme: nmrPipe -in test.fid \ | nmrPipe -fn SP -off 0.5 \ | nmrPipe -fn FT -di \ | nmrPipe -fn TP \ | nmrPipe -fn SP -off 0.5 \ | nmrPipe -fn FT \ | nmrPipe -out test.ft can easily have a final phase-correction (function PS) added just by inserting an additional line: nmrPipe -in test.fid \ | nmrPipe -fn SP -off 0.5 \ | nmrPipe -fn FT -di \ | nmrPipe -fn TP \ | nmrPipe -fn SP -off 0.5 \ | nmrPipe -fn FT \ | nmrPipe -fn PS -p0 90.0 \ | nmrPipe -out test.ft USING VERBOSE MODE The example processing scheme above will work silently unless an error occurs; however, in most cases, it is useful to design processing schemes which generate status messages during processing. This can by done by including the -verb flag with selected instances of nmrPipe. The -verb flag will cause nmrPipe to print status messages showing how many vectors have been processed so far. Usually, the -verb flag is used for one and only one instance in each dimension of a processing scheme: nmrPipe -in test.fid \ | nmrPipe -fn SP -off 0.5 \ | nmrPipe -fn FT -di -verb \ | nmrPipe -fn TP \ | nmrPipe -fn SP -off 0.5 \ | nmrPipe -fn FT \ | nmrPipe -fn PS -p0 90.0 -verb \ | nmrPipe -fn TP \ | nmrPipe -fn POLY -auto -verb \ | nmrPipe -out test.ft If the -verb flag is used for more than one instance in a given dimension, the status messages from the different pro- cessing functions will overlap and replace each other, since the functions in a given dimension are operating at the same time. PROCESSING FUNCTIONS The following is a brief list of processing functions avail- able in the current version of nmrPipe: NULL Null Function: the default nmrPipe function; this leaves the data unchanged. FT Fourier Transform: applies a forward or inverse Complex Fourier Transform to complex, real, or sequential (Bruker) vectors of any size. RFT Real Transform: applies a Real Fourier Transform to real data. Note that this is seldom used; Complex Transforms of real data are more usual. HT Hilbert Transform: adds or replaces imaginary data. Mirror Image Hilbert transforms are also available to accommodate data with half dwell-time delays. MC Modulus Calculation: replaces complex data with its modulus. TP 2D Transpose XY->YX: exchanges the rows (X-Axis) and columns (Y-Axis) for real, complex, and hypercomplex 2D input planes. The 2D transpose function can also be invoked as TP. ZTP 3D Transpose XYZ->ZYX: exchanges rows (X-Axis) with Z-Axis vectors of 3D or 4D input streams. It provides an in-memory alternative for 3D processing; its use is described in the xyz2pipe manual pages. ZF Zero Fill: pads the data with zeros; the zero pad count can be selected by the user, or a suitable value can be selected automatically. In inverse mode, the ZF func- tion reverses effects of zero fill by extracting the original time-domain data. REV Reverse Data: reverses the order of the points in each vector. PS Phase Correct: applies zero and first order phase correction, with the option of first reconstructing imaginary data via hilbert transform. In inverse mode, the PS function negates the phase parameters before applying a phase correction. The phase correction func- tion also includes options for convenient frequency shifting via time-domain correction. The shift parame- ter can be specified in terms of points or spectral units such as PPM, and the chemical shift calibration information is updated accordingly. EM Exponential Multiply: multiplies the data by an exponential window function, with the option of multi- plying the first point of each vector by a given con- stant. In inverse mode, the EM function divides the data by the given window function and first point scal- ing factor. GM Lorentz-to-Gauss: multiplies the data by a mixed exponential-Gauss window function, with the option of multiplying the first point of each vector by a given constant. In inverse mode, the GM function divides the data by the given window function and first point scal- ing factor. TM Trapezoid Multiply: multiplies the data by an tra- pezoidal window function, with the option of multiply- ing the first point of each vector by a given constant. In inverse mode, the TM function divides the data by the given window function and first point scaling fac- tor. SP Adjustable Sine Bell: multiplies the data by an adju- stable sine window, with the option of multiplying the first point of each vector by a given constant. In inverse mode, the SP function divides the data by the given window function and first point scaling factor. ZD Zero Diagonal: sets a range of points on or near the diagonal to zero; it is used in diagonal suppression schemes. MIR Mirror Image: performs various mirror image operations on the input data vectors. EXT Extract Region: extracts a given region of the spectral data; often used to discard empty regions of a spectrum during processing. The extract region can be specified in terms of general regions (left middle or right), specific point locations or chemical shifts. It accu- rately exploits and updates chemical shift calibration information. CBF Constant Baseline: applies a zero-order baseline correction calculated by averaging a portion of each slice. This is commonly used to remove constant offsets from time-domain data. Baseline parameters can be specified in terms of points or spectral units such as PPM. FSH Frequency Shift: applies a frequency shift via inverse Fourier processing and phase correction. The shift can be a non-integer number of points. The shift parameter can be specified in terms of points or spectral units such as PPM, and the chemical shift calibration infor- mation is updated accordingly. MAC MACRO Processing: applies a user-written processing function to the data vectors. The processing function is written in the subset of the C programming language supported by the nmrPipe utility M, a simple C language interpreter. This macro processing facility provides direct access to the data vectors and individual data points without the need to create, compile, and link a complete program. It also provides a large library of vector processing functions, including all the spectral processing functions available to nmrPipe. LP Linear Prediction: applies Complex Linear Prediction to predict or replace data at the head, tail, or interior of the data vectors. Forward, backward, and mixed Forward-Backward modes are supported, along with mirror image modes. DX Derivative: calculates the numerical derivative of the data vectors by central differences. INTEG Integral: calculates the numerical integral of the data vectors by simple summation. SOL Solvent Filter: applies a time-domain convolution func- tion for solvent subtraction. BASE Linear Corrections: applies a series of linear correc- tions to user-selected regions of the input vectors. Baseline parameters can be specified in terms of points or spectral units such as PPM. POLY Polynomial Subtract: subtracts a polynomial from time- or frequency-domain data. In the time-domain, it can be used for solvent subtraction. In the frequency-domain, it can be used for manual or automated baseline correction; the automated baseline correction is very effective. Baseline parameters can be specified in terms of points or spectral units such as PPM. SMO N-Point Smooth: applies a convolution filter for smoothing the data. Smoothing width can be specified in terms of points or spectral units such as PPM. CS Circular Shift: applies a circular shift by an integer number of points, with the option of inverting the sign of shifted points. The shift parameter can be speci- fied in terms of points or spectral units such as PPM, and the chemical shift calibration information is updated accordingly. RS Right Shift: applies a right shift by an integer number of points, replacing shifted points with zeros. The shift parameter can be specified in terms of points or spectral units such as PPM, and the chemical shift calibration information is updated accordingly. LS Left Shift: applies a left shift by an integer number of points, replacing shifted points with zeros. The shift parameter can be specified in terms of points or spectral units such as PPM, and the chemical shift calibration information is updated accordingly. SHUF Shuffle Data: performs various data reordering opera- tions, including separation or interleaving of real and imaginary data, swapping of real and imaginary data, and four-byte swapping. SIGN Sign Manipulation: performs various sign manipulations, including negation of all or alternating points, nega- tion of imaginary data, or negation of the left or right half of each vector. SET Set Data to Constant: sets a range of points in each data vector equal to a specified constant. Range parameters can be specified in terms of points or spec- tral units such as PPM. ADD Add a Constant: adds a specified constant to a range of data points in each vector. Range parameters can be specified in terms of points or spectral units such as PPM. MULT Multiply Data: multiplys a range of data points in each vector by a specified constant. Range parameters can be specified in terms of points or spectral units such as PPM. QART Quad-Artifact Reduction: reduces quad-artifacts by a complex scaling of time-domain data. This implementa- tion will automatically inverse-transform frequency-domain data to apply the scaling. When used via the graphical interface nmrDraw, it can also automatically estimate scaling parameters. COAD Co-Addition of Data: reduces the data by co-adding points in rows or columns, according to specified mix- ing coefficients. OPTIONS The nmrPipe program has two kinds of command-line options; general options, which pertain to any instance of nmrPipe, and function-specific options, which pertain only to selected processing function as specified by the -fn fnName argument. Only the general options are listed below; use nmrPipe with -help and related flags for details on function-specific options. Note that many options are not for general use; some are used for testing and development purposes, while others are used by programs which invoke nmrPipe to perform processing. RULES FOR COMMAND-LINE OPTIONS Command-line arguments play an important part in the use of nmrPipe and related programs. So, it is helpful to list the rules used to interpret a given command line, along with some illustrations. These rules pertain specifically to nmrPipe and the other programs in the NMRPipe System: Command-line arguments and flags are case-sensitive. Command-line options for can be given in any order. Unknown command-line arguments will be ignored. If an option is used more than once on the command-line, the first (leftmost) option will be used, and the remaining ones will be ignored, According to the rules, the following commands perform the equivalent action, since options can be given in any order: 1. nmrPipe -out test.ft -fn FT -in test.fid 2. nmrPipe -in test.fid -out test.ft -fn FT 3. nmrPipe -fn FT -in test.fid -out test.ft Since nmrPipe and related programs will ignore repeated or unknown command-line arguments without generating an error message, commands like the following are valid, although they may not produce the intended results: 1. nmrPipe -in test.fid -out test.ft -fn FT -c 0.5 2. nmrPipe -in test.fid -out test.ft -FN FT 3. nmrPipe -in test.fid -out test.ft -in FT 4. nmrPipe -in test.fid -out test.ft -FT In command 1, the option "-c 0.5" is used, although the FT function does not use a "-c" flag. The "-c 0.5" will be ignored. In command 2, the option "-FN" is used, rather than the correct lower-case option "-fn"; the function specification will be ignored, and the NULL function (no change to data) will be applied instead. In command 3, the function is specified as "-in FT", rather than as "-fn FT" (a typographical error); the function name will be ignored, and the NULL function will be applied instead. In command 4, the "-fn" flag was omitted by accident; the "-FT" will be ignored, and the NULL function will be applied instead. FILE AND PROCESSING FUNCTION OPTIONS -fn fnName Specifies the name of the processing function which nmrPipe should apply; by default, this is the NULL function, which leaves the data unchanged. The name of the processing function is not case-sensitive. More information about individual processing functions can be obtained with the -help and -allHelp options. -in inName Specifies the name of the input file which nmrPipe will read and process, as an alternative to reading data from standard input. Generally, this option will be used at the head (first part) of a processing pipeline. The given input file inName must already exist, and the input data must be in a valid nmrPipe data format. -out outName Specifies the name of the output file where nmrPipe will write results, as an alternative to writing data to the standard output. Generally, this option will be used at the tail (last part) of a processing pipeline. If the given output file outName already exists, nmrPipe will abort without processing unless either the -ov or -inPlace options have also been specified. -ov When used with the -out outName option, this flag will permit over-write of an existing data file. When the -ov flag is not used with the -out outName option, nmrPipe will abort without processing if the given out- put file already exists. -inPlace When used with the -out outName option, this flag allows in-place processing, i.e. a processing pipeline where the input data file is the same as the output data file. Note that if the -inPlace flag is not used in these situations, nmrPipe may create empty or incorrect results without issuing any error or warning messages. The -inPlace flag implies the -ov flag. DATA TYPE AND FORMAT OPTIONS -xi -dt -di Delete Imaginary Data After Processing These flags are used to remove imaginary data from the nmrPipe output. Note that in most processing schemes, imaginary data for a given dimension is removed during processing, once Fourier Transform and phase correction have been performed. For historical reasons, there are two options to delete imaginary data, depending on whether 2D output planes are complex in one or both dimensions; a third option is provided which automati- cally chooses the correct case for a given data stream: -xi Delete Imaginaries; Complex Data Only. This dis- cards the existing imaginary data, so that the 2D out- put planes are all-real. -dt Delete Imaginaries; Hypercomplex Data Only. This deletes the imaginary data from the current dimension (the X-dimension), so that the 2D output planes are complex. This has the effect of reducing the size of the next dimension (the Y-dimension) by a factor of two. -di Delete Current Imaginaries; this is equivalent to -xi in the case of Complex Data, and -dt in the case of Hypercomplex Data. Recall that when the imaginaries of hypercomplex data are deleted, the Y-Axis size is reduced by a factor of two. Note that the -xi and -dt flags are available for backward-compatibility with earlier versions of nmrPipe; the -di flag should be used instead. Only one of these flags may be used for a given invocation of nmrPipe. Some nmrPipe processing functions, such as the 2D Transpose function TP, cannot be used with these flags; incompatible functions will print an error mes- sage. -ai -ad -ac Add Space for Imaginary Data After Processing These flags are used to add space for imaginary data in the output. Note however that they do not actually generate correct imaginary data; they only change the recorded data type and pad the data. Some nmrPipe processing functions, for instance the Hilbert Transform function HT, generate complex data from real data. These functions will automatically change the data type and pad the data without the need for any special flags. However, these functions will not generate hypercomplex from complex data automati- cally; instead, they will just replace the existing imaginary data with new imaginary data. Therefore, these flags are used primarily in situations where hypercomplex data is being reconstructed, for instance in two-dimensional inverse-Fourier processing schemes. -ai Create Imaginaries; Real Data Only. This adds space for imaginary data, resulting in complex 2D planes; the imaginary part of the result will be zeros. -ad Create Hypercomplex Imaginaries; Complex Data Only. This adds space for imaginary data in the next dimen- sion (the Y-dimension); it will increase the size of the next dimension by a factor of two by writing each output slice twice. -ac Create Imaginaries; Real or Complex Data. this is equivalent to -ai in the case of Real Data, and -ad in the case of Complex Data. -noRd This flag will keep nmrPipe from reading data during processing; it is intended for use with processing mac- ros implemented with the MAC function, since the macros can optionally perform their own data input. -noWr This flag will keep nmrPipe from writing data during processing; it is intended for use with processing mac- ros implemented with the MAC function, since the macros can optionally perform their own data output. -inSwap This flag will cause nmrPipe to perform four-byte swap operations (byte order is reverse for each four bytes of data) on the input data header before it is inter- preted. It is intended to accommodate data which ori- ginate on computers with byte-swapped architectures. Note that future versions of nmrPipe may perform this operation automatically. -outSwap This flag will cause nmrPipe to perform four-byte swap operations (byte order is reverse for each four bytes of data) on the output data header before it is written to the output destination. It is intended to convert data for use on computers with byte-swapped architec- tures. Note that future versions of nmrPipe may accom- modate byte-swapped data automatically. -select y1 yn . . . This option can be used to select a region of vectors to processed or to be passed through without process- ing. If used with the -outside option, only vectors outside the selected region will be processed, and vec- tors inside the selected region will be passed through. If used with the -inside option (default), only vectors inside the given region will be processed. The region limits are specified as a Y-Axis lower and upper bound, followed by optional Z-Axis and A-Axis lower and upper bounds. The region limits can be specified in terms of points (no labels) or with labels "hz", "ppm", "%", etc. Note that the -select option is limited to simple functions which do not change the data sizes or types; furthermore it may not work with macro functions which perform their own I/O, or in parallel schemes. -fdata loc1 parm1 . . . The nmrPipe data format includes a header array of 512 four-byte floating point values at the beginning of the data stream. The -fdata option allows adjustment of individual file header values in the output, specified as header location/header value pairs; the locations are specified as origin=1, and the values are floating point. The locations, their meanings, and values are outlined in sethdr. For example, -fdata 222 0.0 would set the location 222 (the 2D Transpose Flag) to zero (Not Transposed). The -fdata option is intended primarily for work-around and testing purposes. REPORT GENERATING OPTIONS -verb [verboseLevel] This flag will cause the nmrPipe processing function to print status messages during processing; the messages are written to standard error. By default, nmrPipe processing functions will work silently unless some error occurs. Most nmrPipe processing schemes have the -verb option set for one and only one processing func- tion in a given dimension of the data. If more than one function has the -verb option set, messages will overlap on the screen. The verboseLevel parameter has the following effect: 0 = Work Silently Unless an Error Occurs 1 = Print Function Name and Data Size 2 = Print Messages as 1D Vectors are Processed If the -verb flag is included without a verboseLevel argument, level 2 is assumed. The frequency of mes- sages produced at verbose level 2 can be adjusted by the -inc option. Note that in some cases, printing status messages can take a substantial portion of the overall processing time, so it is sometimes best to minimize the verbose level. -inc verboseIncrement This option adjusts how often status messages are displayed during processing; this only has an effect if the verbose level is set to 2 by the -verb option. By default, status messages are printed for every 16 vec- tors processed. -dbg debugReportLevel This option enables printing of debug messages; it is included for testing and development purposes. -X11 This option will cause nmrPipe to issue certain error or informational messages via an X11 graphical inter- face, rather than typing the message in the terminal window. It is used by graphics application programs such as nmrDraw which invoke nmrPipe to do processing. -help When used alone, the -help option will generate a list of nmrPipe command-line options, followed by a list of available processing functions. After the help text is printed, nmrPipe will exit. When used with the -fn fnName option, the -help flag will generate argument descriptions for the specified processing function. When used with the -allHelp option, the -help flag will generate argument descriptions for all available pro- cessing functions. Note that all help text is written to standard error; in order to re-direct standard error text to a file, use the ">&" idiom: nmrPipe -help >& help.out -allHelp When used with the -help option, the -allHelp flag will generate argument descriptions for all available pro- cessing functions. PARALLEL PROCESSING AND INTER-PROGRAM INTERFACE OPTIONS -client serverListName This option invokes an experimental parallel processing mode which distributes the given processing task to a list of computers on the network. The current imple- mentation provides only static load balancing, which means that it will not compensate for changing speeds of each computer during processing. The argument serverListName is the name of a text file which lists the computers to use along with arbitrary numbers which indicate their estimated relative speeds. In this example, the three computers named spin, spam, and spite are listed; according to the listing, com- puter spite is 4 times faster than computer spin, while computer spam is 2 times faster than computer spin: spite 40 spam 20 spin 10 When using this parallel processing option, the following notes and restrictions apply: The computers should be named exactly as listed by the UNIX command hostname. The user should should be able to reach each computer via the UNIX remote-shell command rsh(1C). The nmrPipe program should be on the user's search path for all computers listed. Due to network access times and the simple load balancing method, only the most time-consuming func- tions (such as Linear Prediction) will benefit from this kind of parallel processing. The computers named should all have access to some common directory, as specified by the -path option; this directory will be used to establish inter-process communication. -path pathName This option specifies the directory path for the text file which an nmrPipe server process will use to estab- lish communications with its client; by default, it is the current working directory. -server portFileName This option specifies the text file which an nmrPipe server process should use to establish communications with its client; it is intended for use by programs which invoke nmrPipe for client-server based parallel processing. -sID serverID This flag is used to associate a server ID number with a given nmrPipe server process; it is intended for use by programs which invoke nmrPipe for client-server based parallel processing. -inSock inSocketPort This flag will cause nmrPipe to read its input data from the existing socket designated by inSocketPort, as opposed to reading data from standard input or from a file. This option is generally used by programs which silently invoke nmrPipe to perform processing, such as the graphics interface nmrDraw. -outSock outSocketPort This flag will cause nmrPipe to write its output data to the existing socket designated by outSocketPort, as opposed to writing data to standard output or to a file. This option is generally used by programs which silently invoke nmrPipe to perform processing, such as the graphics interface nmrDraw. -smt serverMaxTries This option sets the count for the maximum number of times an nmrPipe server or client can re-try to read or write data over a socket connection. The default value of -1 allows an unlimited number of re-tries. This option is intended for testing and debugging purposes. -sto serveruSecTimeOut This option sets the delay time in microseconds between re-tries of client-server requests to read or write data over a socket connection. The default value is 512 uSec. This option is intended for testing and debugging purposes. MISCELLANEOUS OPTIONS -ibuff inputBuffCount This option selects the number of 1D vectors which will be buffered during input; adjusting this value can alter the efficiency of data access, and possibly enhance processing times. By default, nmrPipe chooses its own buffer count. -obuff outputBuffCount This option selects the number of 1D vectors which will be buffered during output; adjusting this value can alter the efficiency of data access, and possibly enhance processing times. By default, nmrPipe chooses its own buffer count. -buff ioBuffCount This option selects the number of 1D vectors which will be buffered during both input and output, as specified by the -obuff and -ibuff options. -mt maxTries This option sets the count for the maximum number of times an nmrPipe process can re-try to read or write data. The default value of -1 allows an unlimited number of re-tries. This option is intended for test- ing and debugging purposes. -to uSecTimeOut This option sets the delay time in microseconds between re-tries of nmrPipe requests to read or write data. The default value is 512 uSec. This option is intended for testing and debugging purposes. -noSig This flag will prevent nmrPipe from handling interrupt signals on its own; it is included for testing and development purposes. -nofs When used with the -in inName or -out outName options, this flag will suppress any File Size Checking. Without the -nofs flag, nmrPipe will attempt to check if the actual input file size matches the size indi- cated by the header, and it will also attempt to check if the output file system has sufficient space to store the result. The -nofs flag is intended for use in situations where the File Size Checking can't be per- formed properly; this may be the case in certain multi-vendor networks. -tty This flag will allow nmrPipe to send or receive data directly from a terminal screen or keyboard, like a normal UNIX filter. If this flag is not given, nmrPipe will abort if either its input or output seems to be directed to a terminal. Note that in almost every case, if nmrPipe is sending or receiving spectral data from a terminal, something is probably wrong with the process- ing scheme (a missing input or output option, or trail- ing blanks after the "\" continuation character). The -tty flag is intended as a work-around in situations where the input/output type can't be determined correctly by the program. EXAMPLES The examples listed here are exclusively for 2D applica- tions; for 3D and 4D applications, see xyz2pipe and pipe2xyz. This example shows typical processing for a 2D States-Mode spectrum: #!/bin/csh nmrPipe -in test.fid \ | nmrPipe -fn SP -off 0.5 -end 0.98 -c 0.5 \ | nmrPipe -fn ZF \ | nmrPipe -fn FT -verb \ | nmrPipe -fn PS -p0 42.0 -p1 -84.0 -di \ | nmrPipe -fn TP \ | nmrPipe -fn SP -off 0.5 -end 0.98 -c 0.5 \ | nmrPipe -fn ZF \ | nmrPipe -fn FT -verb \ | nmrPipe -fn PS -p0 90.0 -p1 0.0 \ -di -ov -out test.ft2 This example shows 2D inverse processing of the data created above, to restore the hypercomplex 2D States-Mode time-domain data; note the use of the option -ad, which allows complex data to be converted to hypercomplex data: #!/bin/csh nmrPipe -in test.ft2 \ | nmrPipe -fn HT \ | nmrPipe -fn PS -p0 90.0 -p1 0.0 -inv \ | nmrPipe -fn FT -verb -inv \ | nmrPipe -fn ZF -inv \ | nmrPipe -fn SP -off 0.5 -end 0.98 -c 0.5 -inv -ad \ | nmrPipe -fn TP -hyper \ | nmrPipe -fn HT \ | nmrPipe -fn PS -p0 42.0 -p1 -84.0 -inv \ | nmrPipe -fn FT -verb -inv \ | nmrPipe -fn ZF -inv \ | nmrPipe -fn SP -off 0.5 -end 0.98 -c 0.5 -inv \ -ov -out test2.fid This example shows typical processing for a 2D TPPI spec- trum, as acquired on a Bruker AM Spectrometer in sequential acquisition mode, with automated baseline correction in the acquisition axis performed after the 2D Fourier processing. Note that both the Bruker and Real Fourier Transform modes reduce the size of the data by two. #!/bin/csh nmrPipe -in test.fid \ | nmrPipe -fn SP -off 0.5 -end 0.98 -pow 2 -c 0.5 \ | nmrPipe -fn ZF \ | nmrPipe -fn FT -bruk \ | nmrPipe -fn PS -p0 -110 -p1 -150 -di -verb \ | nmrPipe -fn TP \ | nmrPipe -fn SP -off 0.4 -end 0.95 -c 0.5 \ | nmrPipe -fn ZF \ | nmrPipe -fn FT -real \ | nmrPipe -fn PS -p0 -31 -p1 62 -di \ | nmrPipe -fn TP \ | nmrPipe -fn POLY -auto \ -out test.ft2 -verb -ov This example shows processing used for a 2D States-Mode NOE spectrum, acquired with sequential (Bruker) detection. It features polynomial solvent filtering, and separate automated baseline corrections for data to the left and right of the solvent signal: #!/bin/csh nmrPipe -in test.fid \ | nmrPipe -fn SHUF -c2ri \ | nmrPipe -fn POLY -time -ord 6 \ | nmrPipe -fn SHUF -ri2c \ | nmrPipe -fn SP -off 0.5 -pow 2 -c 0.5 \ | nmrPipe -fn FT -bruk \ | nmrPipe -fn PS -p0 -14.2 -p1 13.2 -di -verb \ | nmrPipe -fn TP \ | nmrPipe -fn SP -off 0.5 -pow 2 -c 0.5 \ | nmrPipe -fn ZF -auto \ | nmrPipe -fn FT \ | nmrPipe -fn REV \ | nmrPipe -fn PS -p0 14.0 -p1 -24.0 -di -verb \ | nmrPipe -fn TP \ | nmrPipe -fn POLY -auto -xn 5.3ppm \ | nmrPipe -fn POLY -auto -x1 5.1ppm \ -out test.ft2 -verb -ov In this DQF-COSY example, time-domain phase correction is used to shift the solvent signal closer to zero frequency, where it can be removed more effectively by time-domain con- volution. After this, the shift is removed by a second phase correction, and a tentative linear baseline correction is performed after the transform of the acquisition dimen- sion. A more thorough correction is made after the 2D Fourier processing is done: #!/bin/csh nmrPipe -in dqcosy.fid \ | nmrPipe -fn PS -ls 9 \ | nmrPipe -fn SOL \ | nmrPipe -fn PS -rs 9 \ | nmrPipe -fn SP -off 0.5 -end 0.98 -pow 2 -c 0.5 \ | nmrPipe -fn ZF -auto \ | nmrPipe -fn FT \ | nmrPipe -fn PS -p0 255.6 -p1 78.0 -di -verb \ | nmrPipe -fn BASE -nw 3 -nl 0% 5% 95% 100% \ | nmrPipe -fn TP \ | nmrPipe -fn SP -off 0.5 -end 0.98 -pow 2 -c 0.5 \ | nmrPipe -fn ZF -size 2048 \ | nmrPipe -fn FT \ | nmrPipe -fn PS -p0 -90 -p1 0 -di -verb \ | nmrPipe -fn TP \ | nmrPipe -fn POLY -auto -xn 5.3ppm \ | nmrPipe -fn POLY -auto -x1 5.1ppm \ -out test.ft2 -verb -ov This example shows processing for a 2D plane from an HN-detected 3D spectrum, with numerical solvent suppression and extraction of the amide proton region (the left half) of the acquisition dimension: #!/bin/csh nmrPipe -in fid/test001.fid \ | nmrPipe -fn POLY -time \ | nmrPipe -fn SP -off 0.5 -end 0.98 -pow 2 -c 0.5 \ | nmrPipe -fn ZF -auto \ | nmrPipe -fn FT \ | nmrPipe -fn PS -p0 43 -p1 0.0 \ | nmrPipe -fn EXT -left -di -sw -verb \ | nmrPipe -fn TP \ | nmrPipe -fn SP -off 0.5 -end 0.98 -pow 2 -c 0.5 \ | nmrPipe -fn ZF -auto \ | nmrPipe -fn FT \ | nmrPipe -fn PS -p0 -135 -p1 180 -di \ -out hnco001.ft2 -verb -ov This example shows a slightly different processing for a 2D plane from an HN-detected 3D spectrum; in this scheme, a specific range of chemical shifts is extracted from the HN dimension: #!/bin/csh nmrPipe -in fid/test001.fid \ | nmrPipe -fn SOL \ | nmrPipe -fn SP -off 0.5 -end 0.98 -pow 2 -c 0.5 \ | nmrPipe -fn ZF -auto \ | nmrPipe -fn FT \ | nmrPipe -fn PS -p0 43 -p1 0.0 \ | nmrPipe -fn EXT -x1 5.5ppm -xn 11ppm -di -sw -verb \ | nmrPipe -fn TP \ | nmrPipe -fn SP -off 0.5 -end 0.98 -pow 2 -c 0.5 \ | nmrPipe -fn ZF -auto \ | nmrPipe -fn FT \ | nmrPipe -fn PS -p0 -135 -p1 180 -di \ -out hnco001.ft2 -verb -ov This example shows re-phasing of the acquisition dimension of the 2D spectrum hnco001.ft2 produced by the previous example; Hilbert Transformation is used to reconstruct the imaginary data, and the result file replaces the source data: #!/bin/csh nmrPipe -fn TP -in hnco001.ft2 \ | nmrPipe -fn PS -ht -p0 90.0 -di \ -verb -inPlace -out hnco001.ft2 The example shows conversion of the file hnco001.ft2 for use with nmrPipe on a byte-swapped architecture: #!/bin/csh nmrPipe -fn SHUF -bswap -in hnco001.ft2 \ -out hnco001.ft2 -outSwap -inPlace SEE ALSO M(1), bin2pipe(1), bruk2pipe(1), nmrDraw(1), pipe2xyz(1), sethdr(1), showhdr(1), simSpecND(1), simTimeND(1), var2pipe(1), xyz2pipe(1) DIAGNOSTICS All help text, informational messages, and error messages are sent to standard error. Many error messages are intended to be self-explanatory, but some are included for development and testing purposes. Some common problems are listed here: C-Shell: processing scripts must be explicitly identi- fied as C-shell scripts in order to run properly. They should always begin with the following text as the first characters of the first line: #!/bin/csh Continuation Characters: probably the most common problem encountered when using nmrPipe concerns the use of the backslash character "\" to continue a com- mand onto the next line of a shell-script. This use of the backslash character will only work correctly if there are no trailing blank characters at the end of the line. Arguments: nmrPipe will not issue an error message if it encounters an unknown command argument; therefore, another common source of problems are mis-spelled com- mand arguments, command arguments with dash characters "-" in the wrong places, or command arguments with the wrong combination of upper- or lower-case letters. Input/Output: problems here include use of an input file that doesn't exist, use of an output file in a directory that doesn't exist, or insufficient disk space to save a result. Error messages will also be generated if the output file already exists, but the overwrite flag -ov has not been used. Processing Schemes: in some cases, the processing scheme itself may not be appropriate; for example use of the wrong kind of Fourier Transform, or not delet- ing imaginary data where appropriate. When possible, nmrPipe will try to detect such problems, and issue warning messages. Conversion Parameters: if the spectrometer format data has not been converted correctly, processing may com- plete without error messages but still give the wrong result. Pay close attention to the sizes and acquisi- tion modes of all the dimensions when converting the data. LEGAL NOTICE The nmrPipe program and its related software is based on a design originating in the NIH Laboratory of Chemical Phy- sics, NIDDK. It is not to be distributed or modified. (C) F. Delaglio 1995. The nmrPipe program and its related software is provided "as is". The NIH, the Laboratory of Chemical Physics, NIDDK, and the authors make no warranties, either express or implied, as to any matter whatsoever with respect to the software. In particular, any and all warranties of merchantability and fitness for any particular purpose are expressly excluded. In no event will the NIH, the Labora- tory of Chemical Physics, NIDDK, or the authors be liable for any loss of profits, any incidental, special, exem- plary, or consequential damages of any nature whatsoever (including without limitation, loss of use or other com- mercial or research loss) arising out of or relating to the use or performance of the software.