HP Pascal Record Layout Guide Revision 6 23-Feb-2004 Copyright 2006 Hewlett-Packard Development Company, L.P. Confidential computer software. Valid license from HP and/or its subsidiaries required for possession, use, or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license. Neither HP nor any of its subsidiaries shall be liable for technical or editorial errors or omissions contained herein. The information in this document is provided "as is" without warranty of any kind and is subject to change without notice. The warranties for HP products are set forth in the express limited warranty statements accompanying such products. Nothing herein should be construed as constituting an additional warranty. CHAPTER 1 HP PASCAL RECORD LAYOUT GUIDE 1.1 Overview Of Record Layout On OpenVMS Alpha and OpenVMS I64 systems (and to a lesser extent OpenVMS VAX systems), the layout of data can severely impact performance. The HP Pascal compiler has several features to enable you to write Pascal code that will get the best performance on the target system. The remainder of this document describes the different types of record layouts, HP Pascal features to support them, how to get the best performance with your data structures, and how to convert existing code for better performance. This document focuses on records, but arrays also have similar properties. In almost all cases, where record fields are discussed, you can substitute array components. 1.2 Natural Versus VAX Alignment And Enumeration Sizes The compiler has the ability to layout records in two ways. One, laid out as they are on OpenVMS VAX called "VAX alignment" where an object is packed to the nearest byte boundary (and sometimes to the neearest bit boundary), and two, layed out in another way called "natural alignment" where an object is aligned based on its size. VAX alignment is "Fields and components less than or equal to 32 bits are allocated on the next available bit; otherwise they are allocated on the next available byte." Natural alignment is somewhat more complicated, but essentially fields and components are allocated on the next naturally aligned address for their data type. For example, 8-bit character strings should start on byte boundaries; 16-bit integers should start at addresses that are a multiple of 2 bytes (word alignment); 32-bit integers and HP PASCAL RECORD LAYOUT GUIDE single-precision real numbers should start at addresses that are a multiple of 4 bytes (longword alignment); 64-bit integers and double-precision real numbers should start at addresses that are a multiple of 8 bytes (quadword alignment); and so on. For aggregates such as arrays and records, the data type to be considered for purposes of alignment is not the aggregate itself, but rather the elements of which the aggregate is composed. The natural alignment of an aggregate is that which allows all elements of the aggregate to be naturally aligned. Varying 8-bit character strings must, for example, start at addresses that are a multiple of 2 bytes (word alignment) because of the 16-bit count at the beginning of the string. In addition, for records, the size is rounded up to a multiple of its natural alignment. For example, a record with natural alignment of longword has a size that is a multiple of longwords. Both of these record formats are fully documented in the OpenVMS Calling Standard and the Tru64 UNIX Calling Standard. Besides the alignment of record fields and array components, the sizes of the fields and components can also affect performance. For example, accessing a byte or word on older Alpha system requires more instructions than accessing a naturally aligned longword. On non-VAX systems, HP Pascal uses larger allocation for unpacked Booleans and enumeration types to help performance. Datatype Unpacked Size on VAX Unpacked Size on Alpha/I64 ------------------------------------------------------------------ Boolean 1 byte 4 bytes Enumerated types 1 or 2 bytes 4 bytes For compatibility reasons, the size of all data types in PACKED records and arrays are the same for both VAX and natural alignment formats. 1.3 HP Pascal Features Affecting Data Alignment And Size The HP Pascal for OpenVMS compiler has the following DCL qualifiers: /ALIGN=keyword where keyword is either NATURAL or VAX /ENUMERATION_SIZE=keyword where keyword is either BYTE or LONG The HP Pascal for Tru64 UNIX compiler has similar command line options of: -align keyword where keyword is either natural or vax -enumeration_size keyword where keyword is either byte or long HP PASCAL RECORD LAYOUT GUIDE The /ALIGN qualifier and -align option controls the default record format used by the compiler. The /ENUMERATION_SIZE qualifier and -enumeration_size option controls whether the compiler allocates Boolean and enumeration types as longwords or as 1 or 2 bytes. On VAX systems, the default alignment format is "VAX" and the default enumeration size is "BYTE". On non-VAX systems, the default alignment format is "NATURAL" and the default enumeration size is "LONG". Besides the command line options, HP Pascal also has a corresponding pair of attributes that can be used at the PROGRAM/MODULE level and on VAR and TYPE sections to specify the desired alignment format and enumeration size. They are: ALIGN(keyword) where keyword is either NATURAL or VAX ENUMERATION_SIZE(keyword) where keyword is either BYTE or LONG By using these attribute at the MODULE level, you can extract the records into a separate module and create an environment file with the desired alignment format. By using these attributes on VAR or TYPE sections, you can isolate the records in the same source file. 1.4 Optimal Record Layout The optimal record layout is one where all the record's fields are naturally sized on naturally aligned boundaries and the overall record is as small as possible (ie, the fewest number of padding bytes required for proper alignment). On non-VAX systems, the compiler will automatically place all fields of unpacked records on naturally aligned boundaries. On VAX systems, you have to explicitly ask for natural alignment using either a DCL qualifier or the corresponding attribute. To allow the compiler to do this placement, you should refrain from using explicit positioning and alignment attributes on record fields unless required by your application. In addition, the keyword PACKED should be avoid in all cases except: o PACKED ARRAY OF CHARs require the PACKED keyword to be manipulated as strings. Since chars are 1 byte big, using the PACKED keyword does not hurt their performance in any way. o PACKED SETs may perform better than unpacked SETs. For PACKED SETs, the compiler can sometimes allocate fewer bits for the set field or variable. These smaller sets can often be manipulated directly with longword or quadword instructions instead of using a generic run-time library HP PASCAL RECORD LAYOUT GUIDE routine for larger sets. Inside of unpacked records, PACKED SET fields are no slower than unpacked SET fields. The same holds true for variables of PACKED SETs. PACKED SETs of size 32 or 64 bits are the best performing set types; otherwise a multiple of 8 bits improves performance to a lesser degree. Of course, you may still need to use PACKED if you rely on the record layout; such as binary data files or assuming that types like PACKED ARRAY OF BOOLEAN are implemented as bit strings. While the compiler can position record fields at natural boundaries, it cannot minimize the alignment bytes that are required between fields. The calling standard requires the compiler to allocate record fields in the same lexical order that they appear in the source file. For example, type t1 = record f1 : char; f2 : integer; f3 : char; f4 : integer; end; This record will be 16 bytes big. F1 will be a byte field, followed by 3 padding bytes to position F2 at a longword boundary, followed by F2 itself with 4 bytes, followed by F3 as a single byte, followed by 3 more padding bytes to position F4 at a longword boundary, followed finally by F4 itself with 4 bytes. The optimal layout would be, type t2 = record f1,f2 : integer; f3,f4 : char; end; This record is only 12 bytes big. F1 and F2 will be placed on adjacent longword boundaries, then F3 and F4 can immediately follow since they can appear on any byte boundary, followed by 2 padding bytes to round the size of the record up to a multiple of its natural alignment of longword. To achieve the fewest alignment bytes, you should place larger fields at the beginning of the record and smaller fields at the end. In addition, if you have record fields of schema types that have run-time size, you should place those at the very end of the record since their offset requires run-time computation. So, to recap, you can get the optimal record layout by: HP PASCAL RECORD LAYOUT GUIDE o Avoiding the PACKED keyword except for PACKED ARRAY OF CHARs and possibly PACKED SETs; o Avoiding explicit POS or ALIGNED attributes; o Placing larger fields before smaller fields; o And placing fixed-size fields before run-time sized fields. 1.5 Optimal Data Size As mentioned earlier, the Alpha architecture has strong preferences for data size. In early Alpha systems (prior to the EV56 Alpha chip), the smallest data item it can access directly is a 32-bit aligned longword. Data items that are smaller than 32-bits may impose a performance penalty due to the additional instructions required to access them. The compiler will attempt to reorder loads and stores that manipulate adjacent items smaller than 32-bits to minimize the number of memory references required. For performance reasons, the compiler on non-VAX systems will allocate Boolean and enumerated types as longwords in unpacked records or arrays. On VAX systems, you have to explicitly request this with a DCL qualifier or the corresponding attribute. In addition, you should avoid any explicit size attributes on subrange types. While it is true that [BYTE] 0..255 is smaller than 0..255 (which would allocate 4 bytes since it is a subrange of INTEGER), the additional overhead of accessing the byte sized subrange might be far worse than the extra 3 bytes of storage. Using the BIT attribute on subranges is even worse in terms of the extra instructions required to manipulate a 13-bit integer subrange inside a record. Use these attributes only where needed. Newer Alpha systems have single instructions to manipulate byte and word sized data. You can use the /ARCH=EV56 qualifier on OpenVMS Alpha and the -arch ev56 option on Tru64 UNIX to enable the generation of these new instructions. These new instructions will be emulated on older Alpha machines if the proper version of the operating systems are installed. On I64 systems, the compiler will use the available byte and word size instructions by default but still will attempt to combine nearby loads and stores into larger memory accesses. HP PASCAL RECORD LAYOUT GUIDE 1.6 Converting Existing Records When moving code between OpenVMS VAX, OpenVMS Alpha and OpenVMS I64 systems (and also Tru64 UNIX), you probably want to make sure that you are getting the best performance from your new system. To do that, you must use natural alignment on your record types. 1.6.1 Applications With No External Data Dependencies If your application has no external data dependencies (ie, no stored binary data files, no binary data transmitted to some external device, etc.), then the conversion is as simple as: o Use the default natural alignment; o Use the default enumeration size; o Remove any uses of PACKED that aren't needed; o Remove any explicit positioning or size attributes that aren't needed. o Optionally reorder fields to place larger fields before smaller fields. This won't make the record faster, but will make it smaller. Depending on your datatypes, the removal of any PACKED keywords or attributes may have little improvement in performance. For example, a PACKED ARRAY OF REAL is identical in size and performance to an unpacked ARRAY OF REAL. 1.6.2 Applications With External Data Dependencies If your application has external data dependencies, the process will be more involved since you have to isolate and understand the dependencies. The first step when porting the code would be to use DCL command line qualifiers to select the behavior of the old system. When moving from OpenVMS VAX to OpenVMS Alpha, the steps are: o Use the /ALIGN=VAX qualifier; o Use the /ENUMERATION_SIZE=BYTE qualifier; HP PASCAL RECORD LAYOUT GUIDE o Use the /FLOAT=D_FLOAT qualifier (if you have any DOUBLE binary data); o Leave the code exactly as is. This should produce the exact same behavior on the OpenVMS Alpha or OpenVMS I64 system as you had on your OpenVMS VAX system (with the small difference that using D_Floating data on Alpha systems only give 53 bits of mantissa instead of 56 bits as on VAX systems). When moving from OpenVMS VAX to OpenVMS I64, the steps are: o Use the /ALIGN=VAX qualifier; o Use the /ENUMERATION_SIZE=BYTE qualifier; o Use the /FLOAT=D_FLOAT qualifier (if you have any REAL, SINGLE or DOUBLE binary data); o Leave the code exactly as is. This should produce the exact same behavior on the OpenVMS I64 system as you had on your OpenVMS VAX system (with the small difference that F_floating and D_floating support on I64 systems is done by converting the VAX floating to IEEE floating, performing the operation, and converting back to VAX floating. Some loss of minor loss of precision is unavoidable). When moving from OpenVMS Alpha to OpenVMS I64, there are fewer steps: o Use the /FLOAT=G_FLOAT qualifier (if you have any REAL, SINGLE or DOUBLE binary data). o Lave the code exactly as is. This should produce the exact same behavior on the OpenVMS I64 system as you had on your OpenVMS Alpha system (with the small difference that F_floating and G_floating support on I64 systems is done by converting the VAX floating to IEEE floating, performing the operation, and converting back to VAX floating. Some loss of minor loss of precision is unavoidable). Also, if you are moving from a OpenVMS VAX or OpenVMS ALpha system to a Tru64 UNIX system, the Tru64 UNIX system doesn't support any of the VAX floating point types. If you have binary floating data you want to move from a VAX system to an Tru64 UNIX system, the floating data will have to be converted by some other process into IEEE S_Floating or T_Floating data. You then have to identify which records in your program have external data dependencies. These include binary files (ie, FILE OF xxx), shared memory sections with other programs, binary information passed HP PASCAL RECORD LAYOUT GUIDE to a library routine (like a OpenVMS itemlist, etc.), etc. For the ones without external data dependencies, you can immediately begin to convert them into optimal format (ie, remove any unneeded PACKED keywords and attributes as described earlier). For the ones with external data dependencies, you need to classify them further into: o Records that cannot be naturally aligned due to a hard dependency that cannot be changed (like a record that maps onto an external piece of hardware, a record that is passed to some software you cannot change, etc.). o Records that can be changed after conversion of binary data or cooperating software. For records that you cannot change, isolate them into their own environment file using /ALIGN=VAX, /ENUM=BYTE, and /FLOAT=x_FLOAT (where x_FLOAT is one of D_FLOAT, G_FLOAT, or IEEE_FLOAT). You can also place the ALIGN and ENUMERATION_SIZE attributes on the TYPE or VAR sections that define these records. In that case, you need to also change any uses of the SINGLE, REAL, or DOUBLE datatype to the x_FLOAT datatype to ensure that the proper floating format is used (where x_FLOAT is one of F_FLOAT, S_FLOAT, D_FLOAT, G_FLOAT, T_FLOAT, or X_FLOAT). You don't have to isolate the record if it uses the PACKED keyword since PACKED records are identical regardless of the /ALIGN or /ENUM qualifiers, but isolating the records with dependencies might be useful in the future if you eventually are able to change the format. For records that you might change, you then need to decide whether it is worth your trouble to convert the record and any external binary data. If the record is of low-use and you have a large quantity of external data, the cost of conversion is probably not worth the trouble. If a record is of high-use, but is mostly aligned, then the conversion also may not be worth the trouble. However, a high-use record that is poorly aligned might suggest conversion of external data regardless of the amount of effort required. There are two types of poorly aligned records. One type is records that use the PACKED keyword. PACKED records will layout the same with either setting of the /ALIGN or /ENUMERATION_SIZE qualifiers. To get natural alignment, you must remove the PACKED keyword. However, the keyword PACKED by itself does not guarantee poor alignment. For example, type t = packed record f1,f2 : integer; end; HP PASCAL RECORD LAYOUT GUIDE This record is well aligned with or without the PACKED keyword. It is also well aligned with /ALIGN=NATURAL and /ALIGN=VAX. You can remove the PACKED keyword for completeness, but nothing else needs to be done. The second type are unpacked records that would layout differently with /ALIGN=NATURAL and /ALIGN=VAX. These records will automatically be well aligned by the compiler when recompiled with /ALIGN=NATURAL. Note however that some unpacked records are already well aligned with both alignment formats. For example, type t = record f1,f2 : integer; end; This unpacked record is well aligned with /ALIGN=NATURAL and /ALIGN=VAX. Nothing else needs to be done to this record. The HP Pascal compiler has been enhanced with two new features to help you identify poorly aligned records and how often they are used. The first feature is the /USAGE=PERFORMANCE (on Tru64 UNIX, -usage performance) command line option. This option causes the compiler to generate messages for declarations and uses of record fields that are poorly aligned or poorly sized. For example, given the following program: program a; type r = packed record f1 : boolean; f2 : integer; end; begin end. the compiler can highlight the following: $ pascal/usage=performance test.pas f1 : boolean; .........^ %PASCAL-I-COMNOTSIZ, Component is not optimally sized at line number 4 in file DISK$:[DIR]TEST.PAS;32 f2 : integer; .........^ %PASCAL-I-COMNOTALN, Component is not optimally aligned at line number 5 in file DISK$:[DIR]TEST.PAS;32 %PASCAL-S-ENDDIAGS, PASCAL completed with 2 diagnostics HP PASCAL RECORD LAYOUT GUIDE In this example, the Boolean field in the PACKED ARRAY is only 1 bit big. Single bit fields require additional instructions to process. Also, the integer field is not aligned on a well-aligned boundary for the target system. The /USAGE=PERFORMANCE qualifier gives performance information customized to the target system. For example, on an OpenVMS VAX system, INTEGERs need only be aligned on a byte boundary for "good" performance whereas on an OpenVMS Alpha or OpenVMS I64 system, INTEGERs should be on a longword boundary. The second feature is the /SHOW=STRUCTURE_LAYOUT (on Tru64 UNIX, -show structure_layout) command line option. This option causes the compiler to generate a structure layout summary in the listing file. This summary gives size and offset information about variables, types, and fields. It also flags the same information as the /USAGE=PERFORMANCE command line option. For example, compiling the above program with $ pascal/list/show=structure_layout test.pas produces the following in the listing file Comments Offset Size ----------- ----------- ----------- 5 Bytes R {In PROGRAM A} = PACKED RECORD Size 0 Bytes 1 Bit F1 : BOOLEAN Align 1 Bit 4 Bytes F2 : INTEGER END This section shows the size of the record "R" as well as the sizes and offsets of the records fields. It also highlights any components that were poorly sized or poorly aligned. You can also use the SHOW SYMBOL/TYPE and EVAL/ADDRESS command in the OpenVMS symbolic debugger to get some additional information on record layouts. For PACKED keywords, you can compile with and without the PACKED keyword to see if the fields are positioned at the same offsets or not. So you have now classified the records with external data dependencies into: o Records that are well-aligned with both alignment/enumeration formats o Records that are poorly-aligned, but are not worth converting o Records that are poorly-aligned, but are worth converting HP PASCAL RECORD LAYOUT GUIDE For the well aligned records, no additional work is needed now, but you have to be aware that you still have an external data dependency that might cause trouble if you add fields to the record in the future. For records that aren't being converted now, isolate them into the same environment file or TYPE or VAR sections that you placed the records that you could not convert. For records that are worth converting, you need to plan how to convert the external binary data or cooperating software. For cooperating software, you need to ensure that it gets modified so it views the record with the "natural" layout. You can determine the layout by using the /SHOW=STRUCTURE_LAYOUT command line option described above. For binary data, you need to write a conversion program. Converting existing binary data involves writing a program that reads in the existing data into a poorly aligned record, copies the data into a well aligned record, and then writes out the new record. A simple conversion program would look like: program convert_it(oldfile,newfile); [align(vax),enumeration_size(byte)] type oldtype = packed record { Existing record fields } end; type newtype = record { Record fields reorganized for optimal alignment } end; var oldfile = file of oldtype; newfile = file of newtype; oldvar : oldtype; newvar : newtype; begin reset(oldfile); rewrite(newfile); while not eof(oldfile) do begin read(oldfile,oldvar); { For each field, sub-field, etc. move the data } newvar.field1 := oldvar.field1; newvar.field2 := oldvar.field2; write(newfile,newtype); end; close(oldfile); HP PASCAL RECORD LAYOUT GUIDE close(newfile); end. Notice the "type" keyword before the definition of the "newtype" type. This is very important. Without this keyword, "newtype" would be in the same type definition part as "oldtype" and would be processed with the same ALIGN and ENUMERATION_SIZE settings. When converting data from OpenVMS VAX to OpenVMS Alpha, if you have embedded DOUBLE data, you must use the D_FLOAT predefined type in the "oldtype" definition since the default on OpenVMS Alpha is for G_Floating format. In addition, the compiler will not allow a simple assignment of a D_FLOAT value into a G_FLOAT or T_FLOAT variable. You will need to use the CNV$CONVERT_FLOAT routine provided with OpenVMS to convert the floating data. When converting data from OpenVMS VAX or OpenVMS Alpha to OpenVMS I64, if you have embedded REAL, SINGLE, or DOUBLE data, you must use the F_FLOAT, D_FLOAT, or G_FLOAT predefined types in the "oldtype" definition sine the default on OpenVMS I64 is for IEEE_Floating format. An on OpenVMS Alpha, the compiler on OpenVMS I64 will not allow a simple assignment of a D_FLOAT value into a G_FLOAT or T_FLOAT variable or a simple assignment of a F_FLOAT value into a S_FLOAT variable. You will need to use the CNV$CONVERT_FLOAT routine provided with OpenVMS to convert the floating data. On Tru64 UNIX systems, you can use the cvt_ftof() routine to convert any embedded VAX format data into IEEE format.