How HyperTransport Handles Configuration Accesses

While HyperTransport configuration cycles are compatible with PCI, there are some important differences.

Configuration Cycles Are Memory Mapped

To generate a configuration space read or write, a HyperTransport bridge simply sends a RdSized or non-posted WrSized request using a reserved address range in the 40-bit HyperTransport memory map. This 32MB range, recognized by all devices, is shown in Figure 13-2 on page 312.

Figure 13-2. Configuration Space In The HyperTransport Address Map

How The 32MB Configuration Area Is Used

The 32MB HyperTransport memory map address space reserved for configuration cycles is used to access the 256 byte configuration space of each function in each device on each bus. How the address range is interpreted and how a particular device can recognize configuration cycles it should claim vs. those it must forward is illustrated in Figure 13-3 on page 313 and described below.

Figure 13-3. Configuration Type 0 And Type 1 Request Packet Format

Upper 16 Address Bits Indicate Type 0 And Type 1 Cycle

As in PCI configuration cycles, HyperTransport requires two variants of configuration read/write cycles, type 1 and type 0. The type 0 configuration is generated by a bridge when the cycle has reached the target bus (chain) where the device being accessed resides; the type 1 cycle is in transit to the target bus and should be forwarded by bridges or tunnels in the target path . The bridge to the destination bus will convert it to type 0.

Because HyperTransport configuration cycles are distinguished from other read/write requests only by the fact they target the 32MB reserved configuration address range, the first problem is how to distinguish type 1 from type 0 cycles. The 32MB configuration address range is further divided into two parts : request packets carrying addresses in the upper 16MB of the range are type 1 cycles; requests with addresses in the lower 16MB are type 0 cycles. Referring to Figure 13-3 on page 313, note the following:

HyperTransport Type 1 Configuration Cycle (See Figure 13-3)

If a SizedRD or SizedWt request carries an address with the upper 16 bits set = FDFFh, then the cycle is a type 1 configuration request. Only bridges are allowed to accept these requests, and only if the bus number field in the address (bits 23:16) falls into the range defined by the bridges Secondary-Subordinate bus number registers. The bridge then passes the request downstream.

HyperTransport Type 0 Configuration Cycle (See Figure 13-3)

If a SizedRD or SizedWt request carries an address with the upper 16 bits set = FDF8h, then the cycle is a type 0 configuration request. This will be claimed by the device that also has a match when the device number field (bits 15:11) in the address matches one of its UnitIDs. It then uses the function numbe r and Dword fields to target the particular internal function and configuration space offset.

No IDSEL Signal Needed In HyperTransport

Finally, there is no IDSEL signal to accompany a type 0 configuration cycle in the HyperTransport protocol. The need for this signal has been eliminated because a Base UnitID field has been included in the HyperTransport advanced capability register block so that a device is programmed to "know" its UnitID number(s). This allows the device to decode its own configuration cycles rather than depending on the upstream bridge to do it with IDSEL.

Example: HT Configuration Space Access

The following diagram, Figure 13-4 on page 315, depicts the movement of a configuration request through HyperTransport chains. In the following example, assume the CPU is targeting the configuration space in HyperTransport UnitID 2 on bus 1.

Figure 13-4. HyperTransport Type 1 And Type 0 Configuration Cycles

Events In HT Configuration Example (see Figure 13-4)

1. Low level software executing on the CPU requires access of the configuration space in Device 2 on Bus (chain) number 1.

2. The Host Bridge checks its secondary bus number register, recognizes the target bus is not its secondary bus, and sends a request packet for type 1 configuration cycle onto bus 0 (using the upper half of configuration address range).

3. The HT-to-HT bridge on Bus 0 checks the bus number field in the request and compares it with its own secondary, and subordinate bus numbers . Because the target bus is below it, the HT-to-HT bridge forwards the configuration cycle onto bus 1; at the same time it converts it to a type 0 because the target bus has been reached. Conversion to type 0 simply means shifting the configuration address into the lower half of the configuration address range. Note that the bus number field is stripped off when the cycle is converted to type 0 (see Figure 13-3 on page 313).

4. Device 1 claims the cycle because it is a type 0 configuration cycle AND it carries a device number which matches one of its assigned UnitIDs.

5. Device 1 then uses function number and dword offset fields in the request packet to target the specific internal function and offset location in its configuration space (refer to Figure 13-3 on page 313).

Initializing Bus Numbers And Unit IDs

One of the first steps in HyperTransport configuration is the initial assignment of bus numbers and UnitIDs for each device and chain in the topology. Using a depth-first search algorithm, enumeration software assigns IDs to each device it discovers; if it finds any HyperTransport bridges, it also assigns the primary, secondary, and subordinate bus numbers so that later configuration cycles may find their way to target buses other than bus 0.

Case 1: A Single Chain With One Host Bridge

In a single chain with only one host bridge, enumeration is fairly simple:

1. Following a reset assertion on a chain, the Base UnitID field in the Slave Command register of each in HyperTransport device is cleared to "0".

2. In addition, reset forces the primary, secondary, and subordinate bus number registers in each HyperTransport bridge and the secondary bus number register in host bridges to "0" as well.

3. The transmitter and receiver interfaces on each link perform the low-level negotiation to determine starting bus width. They also perform the required link initialization sequence. Once synchronization is complete, the Initialization Complete bit in each active Link Control register is set.

4. After link synchronization, each active transmitter issues buffer release (NOP) packets to the corresponding receiver to indicate its own input flow control buffer capacities . Once this is done, each transmitter issues NOPs until configuration starts.

5. The host bridge initializes its UnitID counter so it can start assigning UnitIDs to slave devices it discovers (it reserves UnitID 0 for itself).

6. If the host bridge's Link Control register Initialization Complete and End Of Chain bits indicate that another device is attached to its secondary bus, the host bridge sends a series of configuration cycles to the first device in the chain. These type 0 configuration cycles target Bus 0, Device 0 (UnitID 0), Function 0. Because all devices default to UnitID 0, the first device will claim the cycles. Read cycles will target configuration space locations containing Vendor ID, Device ID, Class Code, Header Type, etc.

7. At some point, the host bridge assigns new UnitID(s) to the device by reading the Unit Count field in the Slave Command Register and then programming (writing) the Base UnitID field with the next available UnitID (1). For devices which request more than one Unit ID, this Base UnitI D is the first in a sequential set. Note that the act of writing the Command register causes the Base UnitID field to be updated and the Master Host bit to be set (indicating the device link which points towards the host bridge). Thereafter, the device uses its new UnitID when claiming configuration cycles, etc. Only a reset or rewriting the Slave Command register causes the Base UnitID field to change.

8. Once all functions in the first device are configured, the host repeats the process to access the next device in the chain. It again uses the configuration cycle attributes of Bus 0, Device 0 (UnitID 0), Function 0. Now, the device which is already assigned as UnitID 1, forwards the transaction downstream because the UnitID in the request (0) does not match. The second device is then programmed as the first one was, but the UnitID(s) assigned to it start where the previous device left off (i.e., UnitID 2).

9. After programming each device, the host bridge checks the End-Of-Chain ( EOC ) bit set in the device's downstream Link Control Register. If this bit is set = 1, the enumeration process for the chain is complete.

Case 2: A HyperTransport Bridge Is Discovered

If the enumeration process on a chain encounters a HyperTransport-To-HyperTransport bridge or a bridge from HyperTransport to a compatible protocol (PCI, AGP, PCI-X), then some additional initialization is needed. A bridge is detected when a read of the Header Type field in the configuration header indicates that the device uses the type 1 header format. Software must program the device in accordance with the type 1 header format which includes:

1. Programming the secondary and subordinate bus number registers with the next available bus number (1). This will allow this bridge to forward and/or convert subsequent configuration cycles targeting the new bus(ses) below the bridge.

2. Setting up the Base Address Registers and other fields in the configuration header in accordance with the protocol being used on the secondary bus (HyperTransport, PCI-X, PCI, etc.).

It is permissible for a HyperTransport bridge to have more than one secondary bus and/or a tunnel interface for its primary bus. Refer to Chapter 16, entitled "HyperTransport Bridges," on page 407 for a description of type 1 bridge configuration.

A Note About Bus Numbering In HyperTransport

Bus numbering in HyperTransport systems makes no distinction between HyperTransport, PCI, AGP, or PCI-X buses. As bridges to other protocols are discovered during enumeration, bus numbers are assigned without regard to the particular protocol. This is illustrated in Figure 13-5 on page 319.

Figure 13-5. Bus Numbering In A Mixed Topology

Case 3: Initializing A Double Hosted Chain

The HyperTransport I/O Link Specification makes special provision for enumeration in double-hosted chain topologies. This case does not really have an analogy in PCI or PCI-X systems. Refer to Figure 13-6 on page 320 as the initialization of a double hosted chain is described.

Figure 13-6. Bus Numbering In Double-Hosted Chains

Only One Master Host Bridge

In a double-hosted chain, one host bridge is considered the master, the other is a slave. The HyperTransport I/O Link Specification does not define the method (board strapping, etc.) by which the two bridges determine their respective roles. The specification does say that the master-slave determination must be made before reset. In Figure 13-6, the master host bridge is shown at the left; the slave host bridge is at the right.

Master Bridge Initialization/Configuration Sequence

Assuming the master bridge has been designated, then the events which occur through reset and into initialization are similiar to the single host bridge enumeration described above, with the following differences:

1. After reset assertion, low-level bus width negotiation, link synchronization, and NOP buffer release packets have been sent, the slave host bridge becomes inactive while the master host bridge performs configuration.

2. The master host bridge initializes its UnitID counter and starts the enumeration procedure described in the previous section. It assigns UnitIDs programs for each device it encounters on the chain, and reserves UnitID 0.

3. The slave host bridge will be discovered at the end of the chain by the master host bridge. This occurs when the master host reads the Command register in the slave host bridge and detects the Type field (upper three bits in this register) set = 001b ” indicating the presence of a Host Command capability block.

4. The master host bridge then programs the Double-Ended bit in both its own Command register and the Command register in the slave host bridge.

When the slave host bridge "wakes up", it will detect the Double-Ended bit set in its Command register and not initiate any configuration cycles of its own. All of the internal devices in the chain will have their Master Host bits set to point towards the master host bridge; this means that all requests they will issue will move "upstream" towards the master host bridge.

Refer to Chapter 16, entitled "HyperTransport Bridges," on page 407 for a description of double-hosted chain topologies, including the method used by software to break sharing chains into non-sharing chains.