 










              FDDI Data Link Development


              By Henry S. Yang, Barry A. Spinney, and Stephen Towning



          Abstract                               and efficiency. The advent
              The fiber distributed              of fiber optics, large-
             data interface (FDDI) data          scale integrated circuits,
             link is based on the ANSI           and related technologies
             X3T9.5 FDDI standards with          makes it possible to
             Digital's enhancements              provide a relatively low-
             to provide greater                  cost, high-speed local
             performance, reliability,           area network (LAN) with
             and robustness. The FDDI            large physical extent
             project team encountered            and connectivity. One
             significant challenges,             LAN standard is the fiber
             including the evolving              distributed data interface
             ANSI X3T9.5 FDDI standards          (FDDI), a 100-megabit-per-
             and the development of the          second token ring that uses
             technology to implement             an optical fiber medium.

             the data link, coupled with          The scope of FDDI spans
             time-to-market pressure.            the data link layer and
             Appropriate considerations          the physical layer. The
             and design trade-offs were          FDDI data link provides its
             made to design complexity,          users with communication
             performance, risk, cost,            services on a multiaccess
             and schedule, when deciding         LAN for transmitting and
             functional partitioning and         receiving frames with best-
             semiconductor technology.           effort delivery service
             Extensive simulations and           (also called the datagram
             a novel test approach               service). The development
             were used to verify the             of the FDDI data link
             algorithms, the functional          encountered several
             models comprising the               significant challenges,
             chips, and the physical             including the instability
             chips themselves.                   of the standard, unproven
             Introduction                        technology and protocols,
                                                 and an order of magnitude
              The proliferation and              increase in speed from the
             importance of distributed           International Standards
             system applications place           Organization (ISO) 8802-
             special requirements on             3 carrier sense multiple
             networks in terms of                access with collision
             topological flexibility,            detection (CSMA/CD) LAN.
             performance, reliability,           The design of the FDDI data
             scalability, robustness,            link involved performance


                          Digital Technical Journal Vol. 3 No. 2 Spring 1991  1

 


           FDDI Data Link Development



             considerations such as
             throughput, latency,
             data integrity, and                  The FDDI data link consists
             reliability.[1,2]                   of an LLC sublayer, a

              In this paper we discuss           media access control
             the development of                  (MAC) sublayer, and
             Digital's FDDI data link            their management. The
             and present some of the             LLC sublayer provides
             key algorithms developed            LLC services, the mapped
             by Digital. We then                 Ethernet service, and
             describe the design and             multiplexing/demultiplexing

             development of the FDDI             services for multiple
             data link technology and            users. The key functions
             its implementation in the           provided by the MAC
             FDDI, focusing on the FDDI          sublayer include the
             data link's very large-             FDDI token ring protocol,
             scale integration (VLSI)            frame transmission and
             chip set. Finally, we               reception, initialization
             describe the development            and error recovery for
             methodology used in                 the token ring, address
             simulation, verification,           recognition and filtering
             and testing.                        on receive, and frame error
                                                 detection. Figure 1 is
          FDDI Data Link Overview                an example of the FDDI
                                                 architecture model, showing
              The FDDI data link provides        a dual attachment station
             an upward multiplexing,             (DAS) or dual attachment
             datagram service to support         concentrator (DAC) with a
             multiple data link users            single data link entity. A
             concurrently within the             DAS or DAC may have zero,
             same computer system. The           one, or two link entities;
             FDDI data link incorporates         two or more physical (PHY)
             the ISO 8802-2 logical              port entities; and control
             link control (LLC) and              of their management. A link
             FDDI standards. Also, the           entity is an instance of
             FDDI data link provides a           data link that contains
             mapped Ethernet service,            an LLC and a MAC entity. A
             defined by the Internet             data link user accesses the
             RFC 1103 standard, to map           data link services through
             an Ethernet frame onto an           the port entity. As shown
             ISO 8802-2 LLC frame for            in Figure 1, multiple data
             transport over the FDDI             link users may use the same
             LAN.[3]                             link entity.

              The FDDI data link uses            entities called MACs. It
             the FDDI MAC protocols to           operates by passing a token
             provide fair access to a            sequentially from MAC to
             multiaccess channel, which          MAC around the ring. Only
             is built of individual              the MAC with the token may
             point-to-point physical             transmit frames onto the
             links.[4] A token ring              ring, and only one token
             consists of an ordered,             can be present on the ring
             cyclic set of MAC protocol          at any instant. At the end

          2  Digital Technical Journal Vol. 3 No. 2 Spring 1991

 


                                                     FDDI Data Link Development



             of transmitting its frames,         and a set of frame-based
             the MAC transmits the               protocols. The set of
             token onto the ring. Frames         frame-based protocols
             circumnavigate the entire           for station management
             ring and are subsequently           includes duplicate address
             removed (stripped) by the           detection, neighbor
             originating MAC after one           notification protocol
             and only one rotation.              for ring map generation,
             The FDDI MAC protocol               and loopback protocol.
             is different from the               The scope of the station
             ISO 8802-5 token ring               management frame-based
             protocol.[5]                        protocols is limited
              The FDDI MAC protocol uses         to a single ring. These
             a timed token protocol,             functions are implemented
             whereby MACs on the token           in the Common Node Software
             ring cooperatively attempt          (CNS) and the FDDI chip
             to maintain a specified             set.[7]

             token rotation time by
             using the observed network
             load to regulate the             FDDI Data Link Algorithms
             amount of time a MAC may
             transmit.[6] The specified           Digital's realization of
             token rotation time is              the FDDI data link includes
             called the target token             major enhancements and
             rotation time (TTRT). The           value-added features. The
             TTRT is negotiated using            development of the FDDI
             a distributed algorithm             data link encountered
             called the claim token              several architecture and
             algorithm, which is invoked         implementation issues. The
             each time the ring is               key issues included design
             initialized. The FDDI MAC           for high performance, error
             protocol also includes              characteristics and data
             fault detection and                 integrity, removal of
             recovery functions to aid           frames by bridges, cleaning
             in the restoration of ring          the ring of unwanted frames
             operation in the presence           and long fragments, and
             of transient faults.                stability and reliability
              As shown in Figure 1,              of the ring.[2] Digital
             each instance of a station          provided the impetus to
             (e.g., DAS or DAC) contains         resolve these issues as
             a set of station management         well as the solutions to
             functions. Some of the              be incorporated into the
             key station management              ANSI FDDI standards. Some
             functions included are              of these solutions are
             initialization, observation         described below.

             and control of link                 Frame Content Independent
             and PHY port entities,              Stripping (FCIS) Algorithm
             control of the insertion
             and removal of the
             station from the ring,
             topology control, fault
             detection and recovery,

                          Digital Technical Journal Vol. 3 No. 2 Spring 1991  3

 


           FDDI Data Link Development



              The underlying logical             of stations in the extended
             topology of all token               LAN.[9] If a bridge were
             rings is a closed loop              to use the source address
             structure, which inherently         match algorithm, the bridge
             has the property of                 would have to complete the
             continuously circulating            address match operation
             frames transmitted on               within one microsecond from
             the ring. Because of                the beginning of the frame
             this property, all token            reception. Therefore, the
             rings have algorithms for           use of the source address
             removing frames transmitted         match algorithm by a
             by MACs on the ring. The            bridge imposes significant
             frame removal algorithm is          costs and implementation
             called a frame stripping            complexity. The design
             algorithm. When stripping           of the frame stripping
             a frame, the fragment size          algorithm is made more
             (remnant of the frame)              difficult by the fact that
             must be less than 17                there can be up to 560
             bytes. Frames that are              frames outstanding (i.e.,
             not properly stripped               transmitted but yet to
             can remain and traverse             be stripped) and the fact
             the ring repeatedly,                that there is less than
             wasting bandwidth and               one microsecond to decide
             resources within systems            whether to strip or to
             on the ring and causing             repeat the frame.
             severe congestion in the             Digital developed an
             systems due to delivery of          algorithm called the
             duplicate frames.                   frame content independent

              The FDDI MAC protocol              stripping (FCIS) algorithm,
             uses a frame stripping              which is implemented in
             algorithm in which all MACs         the MAC chip.[10,11]
             on the ring continually             The algorithm is based
             strip received frames               on stripping the same
             whose source address                number of frames that
             matches their own MAC               the MAC transmitted on
             address. Limitations                the ring independent of
             of this algorithm arise             the content of the frame.
             when implementing bridges           After the MAC captures
             or systems that need to             the token for frame
             transmit frames with                transmission, a local count
             source addresses which              is incremented each time
             are different from each             a frame is transmitted.
             MAC's own address.[8]               After transmitting all
             For example, a bridge               its frames, the MAC
             may forward frames with             transmits a void frame,
             no modifications, and,              which is a minimum size
             therefore, the forwarded            frame (i.e., 17 bytes),
             frames contain source               before transmitting the
             addresses which are                 token.[10,12] On receive,
             different from the bridge           if the count is greater
             address. Also, a bridge may         than zero, the received
             support tens of thousands           frame is stripped; and


          4  Digital Technical Journal Vol. 3 No. 2 Spring 1991

 


                                                     FDDI Data Link Development



             for each error-free frame           transmitting frame Z1,
             received and stripped, the          station B transmits its
             count is decremented. After         void frame, VB, and then
             transmitting the token, the         the token. When station B
             MAC does not strip frames           receives its void frame,
             when the count is equal to          the count is reset to zero,
             zero, except for frames             which causes station B
             with a source address               to stop frame stripping.
             matching the MAC's address.         Subsequently, an entirely
             When receiving an error-            new epoch of transmission
             free void frame with the            and frame stripping can
             source address matching the         begin with the next token
             MAC's address, the count            capture.
             is unconditionally reset             The combination of
             to zero. As a result, the           the count and the void
             algorithm uses a count that         frame provides robust
             is kept locally to track            frame stripping that is
             the number of outstanding           independent of the content
             frames for stripping, and           of the transmitted frame.
             it uses a transmitted void          The FCIS algorithm also
             frame as a backup mechanism         has the desired property of
             to indicate the end of              operating transparently
             stripping.                          with other MACs which

              The operation of the               are not implementing the
             FCIS algorithm is seen              algorithm. Implementing
             in the space-time diagram           the FCIS algorithm in the
             of Figure 2, which shows            MAC chip greatly reduces
             three stations on the ring          the cost and complexity of
             while station B is the              products, which otherwise
             only station participating          may need additional
             in the FCIS algorithm.              hardware components to
             Time increases from top to          perform similar functions.
             bottom, and the position            Ring Purging Algorithm
             of downstream stations
             on the ring goes from                One of the well-known
             left to right in this               problems on a token ring
             diagram. Station A first            is the circulation of
             receives the token T and            frames or long fragments
             transmits frames A1, A2,            that are not stripped
             and the token. Station              by the transmitter. The
             B then transmits frames             frame or long fragment not
             X1, Y1, and Z1, which               stripped by the transmitter
             have source addresses               is called a no-owner frame
             X, Y, and Z that are not            (NOF). On an idle ring,
             the same as station B's             NOFs can circulate around
             MAC address. As shown in            the ring, along with the
             Figure 2, each time station         token, continuously at the
             B transmits a frame, it             speed of the ring. NOFs
             increments its count; and           may be received by one or
             each time station B strips          more systems on the ring
             an error-free frame, it             repeatedly at an extremely
             decrements the count. After         high rate, which can lead


                          Digital Technical Journal Vol. 3 No. 2 Spring 1991  5

 


           FDDI Data Link Development



             to severe congestion and            purge cycle. Once the purge
             waste of system resources.          cycle has started, the
             For example, a single NOF           purger unconditionally
             on an idle ring can create          removes all frames or
             a frame arrival rate of             fragments received. The
             2,700 frames per second             purge cycle is terminated
             (for maximum size frame)            when the purger receives
             to about 290,000 frames             one of its error-free void
             per second (for minimum             frames, a token, or a ring
             size frame). In the best            initialization frame. In
             case, the NOFs are removed          order to increase the
             when they arrive at a MAC           probability of correctly
             that is transmitting (i.e.,         terminating the purge
             holding the token).                 cycle, the purger transmits
              Digital developed an               two void frames; but it
             algorithm called the ring           terminates its purge cycle
             purging algorithm to remove         based on receiving only one
             NOFs. The ring purging              error-free void.

             algorithm ensures that               Figure 3 shows the
             NOFs do not traverse the            operations of the ring
             ring more than twice. The           purger in removing an
             implementation of ring              NOF, during an idle ring
             purging consists of two             and during a busy ring.
             related, but different,             Each time station S3 (the
             algorithms. The first one           purger) receives the token,
             is the purger election              it may transmit its frames,
             algorithm, which is a               followed by two void frames
             distributed election                and then the token. It
             algorithm to select a               purges the ring until it
             designated MAC to be the            receives one of its error-
             purger for the ring. The            free void frames. As shown
             second algorithm is the             in the example, the NOF
             purging algorithm, which is         was purged by station S3
             executed by the designated          on its second traversal
             MAC to clean the ring               around the ring. Also,
             of NOFs. We describe the            the example shows that
             purging algorithm in this           the purging of the ring is
             section.                            transparent (i.e., there is
              The purging algorithm              no disruption to the ring).

             adopted for Digital's
             FDDI data link purges the
             ring transparently each
             time a token is received
             by the purger. When the
             purger receives a token,
             it begins a purge cycle by
             transmitting two special
             frames, called void frames.
             If the purger has frames
             to transmit, it completes
             the transmission of its
             frames before starting the

          6  Digital Technical Journal Vol. 3 No. 2 Spring 1991

 


                                                     FDDI Data Link Development



              The impact of ring purging      FDDI Data Link Chips

             on ring performance is               The FDDI MAC sublayer
             negligible, because the             functions are implemented
             ring purger only initiates          by the three FDDI data
             the purge cycle when it             link chips: the ring memory
             has the right to use a              controller (RMC), media
             token. In the worst case,           access control (MAC), and
             the ring purger's effect            content addressable memory
             on usable bandwidth is              (CAM). The RMC interfaces
             less than 0.22 percent. For         between the frame buffer
             implementations compliant           memory on the system side
             to the ANSI FDDI MAC                and the MAC chip on the
             standard, these void frames         network side. It consists
             have no effect since the            of a direct memory access
             standard prohibits copying          (DMA) engine designed to
             void frames.                        supply the MAC chip with
              The ring purging algorithm         frames to send and to store
             removes NOFs, including             frames received from the
             long fragments, without             MAC chip. The interface on
             disrupting the operation            the system side provides
             of the ring, and it                 gathered reads on transmit
             removes NOFs within two             and scattered writes on
             traversals of the frame.            receive. Although the RMC's
             In addition, it has an              interface to the MAC chip
             important property that             was custom designed for
             permits more than one               FDDI operation, the RMC
             purger to operate in the            can, in principle, be used
             same ring at any time. This         for other data links that
             property allows the purger          run at 100 megabits per
             election algorithm to be            second or less. The MAC
             more optimistic (i.e., when         and CAM chips implement
             in doubt during election,           the FDDI MAC protocol
             one can start purging)              functions. The functions
             during the transition               implemented by the MAC
             period when the distributed         chip include the token
             election algorithm is               access protocols, frame
             stabilizing. The purger             delineation, frame parsing,
             election algorithm is               address recognition, frame
             implemented in the Common           check sequence generation
             Node Software (CNS), and            and verification,
             the purging algorithm is            frame insertion, frame
             implemented in the MAC              repetition, frame removal,
             chip.                               token generation, and
                                                 error detection and
                                                 recovery algorithms (e.g.,
                                                 the beacon and claim
                                                 algorithms). The CAM chip
                                                 provides the destination
                                                 address filtering, which
                                                 determines if a received
                                                 frame is to be received or
                                                 discarded, and the setting


                          Digital Technical Journal Vol. 3 No. 2 Spring 1991  7

 


           FDDI Data Link Development



             of the A-indicator, which
             is part of the frame status
             field.                              buffers. The buffer memory

             RMC Chip                            interface provides a burst
                                                 mode DMA for read/write
              The RMC chip is a high-            from/to the buffer memory
             performance coprocessor             on a single bus arbitration
             intended for full-duplex            cycle, where the burst
             data transfer between the           size can be up to four or
             buffer memory and the MAC           eight longwords. The RMC
             chip. It uses a pair of             can provide a maximum data
             circular buffer queues for          transfer rate of about 44
             transmit and receive to             megabytes per second.
             manage DMA data transfers
             to and from the buffer               The RMC is implemented
             memory. Two independent             using a 1.5-micron-drawn,
             on-chip, first in, first            two-metal-layer custom
             out (FIFO) buffers, for             complementary metal oxide
             receive and transmit, are           semiconductor (CMOS)
             provided to decouple the            technology. It uses roughly
             buffer memory from the              87,000 transistors, a large
             real-time nature of the MAC         number of which are used
             interface. A fragment and           for the two FIFO buffers,
             frame filter is provided to         where the receive FIFO
             reduce unnecessary memory           buffer is 256 bytes and
             accesses caused by the              the transmit FIFO buffer
             reception of fragments              is 128 bytes. The RMC uses
             or frames not addressed             102 signal pins and is
             to this station. As shown           available in a 132-pin
             in Figure 4, the RMC chip           cerquad package. It is
             has three interfaces: the           also a fully synchronous
             processor interface, the            design [some self-timed
             MAC chip interface, and the         logic is used in the FIFO
             buffer memory interface.            random access memory (RAM)
             The processor interface             devices] using a 12.5-
             allows the initialization,          megahertz (MHz) primary
             control, and observation            clock and 25-MHz clock
             of the RMC. The MAC chip            for sampling incoming
             interface consists of high-         signals. The FIFO RAM was
             speed transmit and receive          implemented using a full
             data paths for transfer             custom methodology, and the
             of data and control                 remainder was implemented
             information between the             using an automated standard
             MAC chip and the RMC's FIFO         cell methodology.

              The RMC interfaces to              by the RMC followed by
             buffer memory using a data          a burst of data cycles,
             and address multiplexed             either driven by the RMC
             bus, which is a 32-bit-wide         on receive or driven by the
             bus plus the four parity            buffer memory on transmit.
             signals and additional              One of the unique features
             control signals. A bus              of this chip is that it is
             transaction consists of             able to use a 32-bit-wide
             an address cycle driven             buffer memory composed of

          8  Digital Technical Journal Vol. 3 No. 2 Spring 1991

 


                                                     FDDI Data Link Development



             low-cost [100-nanosecond            one for the frame, then
             (ns) access time] dynamic           the RMC writes the receive
             random-access memory (DRAM)         status and the frame byte
             chips, whereas many of              count into the descriptor.
             the other FDDI memory               Transmit buffers are also
             controllers available on            512 bytes long and the RMC
             the market require a 64-bit         reads the size (in bytes)
             buffer memory or require            from the first descriptor
             very fast DRAM or static            for the frame.
             random access memory (SRAM)         MAC Chip
             chips. To achieve these
             reduced memory and chip              The MAC chip implements
             requirements, the RMC's             the FDDI MAC protocols, and
             buffer memory accesses              it interfaces between the
             were done in bursts of              RMC or equivalent chip
             between one and eight               and the FDDI physical
             longwords. Then, by making          layer chip.[13] As shown
             use of the DRAM's fast page         in Figure 5, the MAC
             mode, in which subsequent,          has four interfaces: the
             sequential reads/writes are         processor interface, the
             faster than the first one,          RMC interface, the physical
             the needed buffer memory            layer chip interface,
             bandwidth is attainable.            and the CAM interface.
              The RMC directly accesses          The processor interface
             a transmit ring and                 allows the initialization,
             a receive ring, each                control, and observation of
             consisting of a circular            the MAC. The RMC interface
             queue of descriptors. Each          is custom designed for FDDI
             descriptor supplies the             operation and allows the
             buffer memory address of            MAC to interface to either
             a transmit buffer or a              the RMC or to equivalent
             receive buffer. An OWN bit          chips implementing the RMC
             mechanism is used in each           interface. The physical
             descriptor to determine             layer chip interface allows
             if the descriptor and its           the MAC to receive and
             buffer is owned by the              transmit on the FDDI ring.

             RMC or not. It supports              The MAC chip is implemented
             gathered read and scattered         using a 1.5-micron-drawn,
             write in which frames to            channelless, two-metal-
             be received or transmitted          layer CMOS gate-array
             can use one or multiple             technology and uses roughly
             buffers (and hence multiple         49,000 transistors (12,000
             descriptors), but a                 used gates). The MAC chip
             specific buffer/descriptor          uses 86 signal pins and
             can only be used by one             is available in either a
             frame. Each receive buffer          120-pin pin grid array
             is required to be 512 bytes         (PGA) package or a 120-
             long. The RMC rewrites each         pin plastic quad flat pack
             descriptor in the receive           (PQFP) package. This fully
             ring to indicate the number         synchronous design uses
             of bytes actually used; and         primarily a single 12.5-MHz
             if the buffer is the last           clock (80-ns cycle time);


                          Digital Technical Journal Vol. 3 No. 2 Spring 1991  9

 


           FDDI Data Link Development



             this operation is applied
             to the microprocessor bus
             as well. An additional
             double-speed clock is used          but exploits more of the
             in some of the peripheral           inherent parallelism of the
             interface logic to sample           algorithm and hence can be
             incoming signals and                implemented using slower
             prevent hold time problems          clock speed.
             associated with clock skew           One of the important
             between different chips.            features provided by the

              The internal structure of          MAC chip is the support
             the MAC chip has a full-            of a 3-byte packet request
             duplex architecture. No             header for transmission.
             logic is shared between             The use of the packet
             the receive and transmit            request header construct
             portions of the chip. Hence         allows simple, pipelined
             this chip can receive and           processing of the transmit
             transmit simultaneously for         descriptor along with the
             an indefinite period. This          transmit data at high
             capability complies with            speed. This construct
             the ANSI FDDI MAC standard          allows a software device
             that implementations be             driver to build a transmit
             able to receive, parse,             descriptor to precede and
             and validate certain frames         identify the frame, which
             (e.g., claim frames and             is then passed through
             beacon frames) even while           the system bus and DMA
             transmitting. Two separate          data movers for delivery
             frame check sequence (FCS)          to the MAC chip. Every
             checker/generators are              frame transmitted by
             required for transmit and           the MAC chip must first
             receive functions. The MAC          contain the packet request
             chip calculates the FCS,            header, which is used as
             which is specified as a             the transmit descriptor and
             specific 32-bit cyclic              is not transmitted as part
             redundancy check, eight             of the frame. The packet
             bits at a time. A one-bit           request header is used to
             implementation is much              instruct the MAC chip on
             smaller, but requires               how and when to transmit
             a 100-MHz clock. Even               the frame. For example,
             with such a clock, it               it instructs the MAC chip
             is not easy to implement            on whether to append the
             one bit at that speed. A            FCS to the frame or not,
             byte-wide implementation            or on the type of token to
             requires considerably more          use when transmitting the
             exclusive OR (XOR) gates,           frame.

             CAM Chip                            multicast addresses used by
              The CAM chip provides a 64-        upper layer protocols and
             entry content addressable           the data link management
             memory where each entry             protocols. The CAM is used
             is a 48-bit address.                to parse the destination
             Typically, the entries              address field of each frame
             in the CAM consist of               received to decide whether

          10  Digital Technical Journal Vol. 3 No. 2 Spring 1991

 


                                                     FDDI Data Link Development



             the destination address
             received matches one of
             the entries in the CAM.
             The result of the address
             match is then provided in
             real time as input to the
             MAC chip, which decides
             whether to receive or
             to discard the frame. In
             the worst case, the MAC
             protocol requires that
             the destination-address-
             match decision be completed
             in less than 10 bytes
             of time (i.e., 800 ns),
             starting from the end of
             the destination address
             field.


              The CAM chip is implemented
             using a 1.5-micron-drawn,
             two-metal-layer custom CMOS
             technology and uses roughly
             44,000 transistors-34,000
             of which are used for the
             core array of 64 words of
             48 bits (plus a valid bit).
             The CAM chip uses 37 signal
             pins and is available in a
             44-pin cerquad package. It
             also is a fully synchronous
             design using the same 12.5-
             MHz primary clock (plus the
             25-MHz clock for sampling
             incoming signals).



















                         Digital Technical Journal Vol. 3 No. 2 Spring 1991  11

 


           FDDI Data Link Development



              As shown in Figure 6,              one transfer or compare
             the CAM consists of three           at a time. The arbitration
             interfaces: the processor           mechanism guarantees that
             interface, the MAC chip             the compare register access
             interface, and the physical         is never delayed, since the
             layer chip interface.               MAC chip requires the match
             The physical layer chip             indication to be valid
             interface consists of an 8-         during a specific clock
             bit bus which transfers             cycle. The CAM is also
             data bytes from the                 designed to allow entries
             physical layer chip to              to be added and removed at
             the MAC chip. Every 80 ns,          any time (even while the
             one byte is loaded into             FDDI ring is running).

             the appropriate position
             in the 6-byte internal           FDDI Simulation and
             compare register. Once           Verification
             the last byte of a 48-bit            Digital's FDDI technology
             address has been clocked            development method was a
             into the CAM chip's compare         top-down approach, starting
             register, a match/no match          with high-level system
             indication is given to the          models of FDDI behavior
             MAC chip within 120 ns by           and progressing to more
             the MAC chip interface.             detailed behavioral and

              The other interface to             structural models as
             the CAM is the processor            confidence in functionality
             interface, which is used            increased. Several
             by the processor to load            simulation models and
             and change the CAM entries.         analytical models were
             The processor does not              developed to study and
             directly read or write the          model the FDDI at the
             CAM array, but instead              architecture and system
             reads and writes (16                levels. Using these models,
             bits at a time) to three            studies were done on the
             16-bit data registers               error characteristics
             and one 16-bit command              and robustness of FDDI,
             register. By clearing               stability of the ring
             the appropriate bit in              topology, performance and
             the command register,               operational behaviors,
             the processor requests a            and correctness of the
             read, write, or compare             protocols.[1]
             from the CAM array. The              Every chip was partitioned
             arbiter ensures that the            into logical subblocks
             CAM array is idle (i.e.,            and a DECSIM (Digital's
             not used by the compare             simulation tools and
             operation) before allowing          language) block-behavioral
             the processor's request             model was developed for
             to complete. In order to            each subblock. The external
             ensure atomicity for read,          interfaces and internal
             write, and compare, this            structure of the model
             arbitration is required             accurately represented some
             because the actual CAM              unit or subunit of a chip.
             array can only perform

          12  Digital Technical Journal Vol. 3 No. 2 Spring 1991

 


                                                     FDDI Data Link Development



             When the model interfaced           between chips and was
             to some nonexistent model           repeated until all chips
             (e.g., unwritten behavioral         in the chip set had been
             model or external interface         tested together as a
             to another chip), a                 system.
             transactor was provided.             After chip layout was
             The transactors provided            completed, the structural
             accurate models of                  models were enhanced to
             interfaces (timing,                 reflect the more accurate
             control, and data                   timing data. The test
             signals), but lacked                vectors were again applied
             the internal detail of a            using a checker model,
             behavioral model. The use           which consisted of one
             of the transactors was              block-behavioral model and
             particularly important              one structural model of
             when modeling the buffer            the same chip. The test
             memory interface, since the         vectors were applied to
             design of the interface is          each model simultaneously,
             implementation dependent            and external and internal
             (designed according to              signals of both models were
             product needs by the                compared for consistency.
             development group using             Any discrepancy between
             the FDDI chip set).                 the monitored signals was

              Each block-behavioral              thoroughly investigated and
             model was tested in                 corrected as necessary.
             isolation; then all were             For test and debug, we
             combined to produce a               planned to develop a
             behavioral model of the             dedicated hardware tester
             target chip. When the chip          to test the physical FDDI
             model was successfully              chips. Unfortunately bugs
             tested, each behavioral             found by using such a
             subblock was replaced by            tester occur too late
             a corresponding structural          in the process-the chips
             model representing gate             are already built. In
             /transistor logic.                  order to meet our time-
             The new chip model was              to-market goal, we needed
             then retested until                 to maximize activity in
             the structural model                the simulation environment.
             behaved identically to its          Rather than waiting for
             behavioral counterpart.             the hardware, we decided
             As the model of each                to develop and apply as
             chip was completed, the             many of the tester-based
             transactors driving its             tests as possible in the
             external interfaces were            simulation environment.
             replaced with the model
             for the next adjacent chip.         Simulation Test Bed
             The resulting combination            The version of DECSIM
             was then tested together            we were using provided
             using the remaining                 a C language interface
             transactors and test                capability. By writing the
             vectors. This process               tests in C language and
             tested interoperability             making use of a common but


                         Digital Technical Journal Vol. 3 No. 2 Spring 1991  13

 


           FDDI Data Link Development



             small environment-specific             (196,560 VAX 11-780 CPU
             interface library, it was              hours equivalent).
             possible to simulate the            o  For the single-node test
             system behavior using the              bed, there were 827 MAC
             chip models. By rewriting              and 384 RMC tests. Using
             the interface library,                 the 8840 cluster, the
             the same tests were run                test suites required
             unchanged on the tester and            336 (2,184 VAX 11-780
             were used in regression                CPU hours) and 192 hours
             testing of the chips                   (1,248 VAX 11-780 CPU
             during the fabrication                 hours), respectively,
             process. The tests,                    for completion.
             interface library, and
             chip models became known             The individual tests varied
             as the simulation test              in complexity from those
             bed. This was the first             requiring a few CPU minutes
             attempt to use this DECSIM          to those requiring days to
             capability as a cornerstone         run. For example, one MAC
             of a development strategy.          test which loops back ten
              All the chip models were           512-byte packets within
             combined to construct an            the single-node test bed
             entity resembling a single          required 36 CPU hours to
             attachment station (SAS)            complete.

             which was then tested                The importance of the test
             as a full system, that              bed cannot be overstressed,
             is, a single operational            as it is the major
             FDDI node (in loopback).            innovation in Digital's
             Extensive use of mixed              development methodology.
             mode simulation (mixing             Some CNS firmware was
             transactors and behavioral          also developed in this
             and structural models)              environment. The benefits
             aided test bed performance          provided include:

             because the level of model          o  The tests assisted
             detail could be varied,                the chip designers
             depending upon the area                to discover bugs in
             being tested. Time was                 the chip designs at
             saved by substituting                  the correct stage of
             higher level models in                 development-in design
             areas peripheral to those              rather than after
             under test.                            silicon.

              The simulation cluster was         o  It was far easier to
             a cluster of four VAX 8840             debug the tests in the
             systems, each system having            simulation environment
             four processors. Some idea             rather than on the
             of the extent of the effort            physical hardware.
             expended can be conveyed by            During simulation
             the following statistics:              we could observe and

             o  The total number of                 control chip behavior.
                CPU hours used for the              Test results were easy
                design and verification             to determine with
                effort was 30,240                   clear pass or fail

          14  Digital Technical Journal Vol. 3 No. 2 Spring 1991

 


                                                     FDDI Data Link Development



                indications, and no               The main value of the
                human interpretation of          tester was its capability
                large strings of 1's and         to perform long-term,
                0's was required.                steady-state testing,
             o  Since all test code              using billions of frames.
                was developed in C,              It was also required to
                no specialized test              test complex topologies
                language was required,           using multiple testers (as
                and standard support             FDDI stations) in large
                tools were readily               rings. These activities
                available. Libraries             are prohibited in the
                of reusable (debugged)           simulation environment
                test-support functions           because of the excessive
                were available,                  amount of compute time
                providing such functions         required. The tester
                as create frame, compare         had to be capable of
                frame, write control             driving the data at
                /status register, and            full FDDI bandwidth,
                read control/status              introducing controlled
                register.                        error conditions on the
             o  Easy transition back             fiber, and accurately
                and forth between the            monitoring activity on
                physical and simulation          the ring at full FDDI
                environments was                 bandwidth. The tester
                important. Any bugs              itself was constructed so
                found in the physical            that it could be controlled
                environment had to               via an external Ethernet
                be reproduced in the             link and multiple testers
                simulation environment           could be synchronized via
                in order to test bug             external clock and control
                fixes.                           lines.

             FDDI Tester                         Testing the First Chips

              In parallel with chip               The first pass of chips
             design, another development         tested free of major
             group was assigned to test          defects. Most tests
             the chip set in a prototype         applied in the test bed
             FDDI system. Their approach         passed the first time;
             was to design an FDDI               the few exceptions were
             tester that used the                due to unrealistic timing
             FDDI chip set. We also              expectations of the tester
             wanted to make this tester          environment.

             configurable as various              The next stage of testing
             FDDI entities, e.g., an             involved combining
             SAS, a wiring concentrator,         testers into multinode
             or an Ethernet-to-FDDI              FDDI configurations and
             bridge, so that we could            exchanging data at full
             use the FDDI tester to              speed over extended
             build an FDDI ring to               periods. This stage
             investigate the behavior            was also successful;
             of the ring.                        no additional bugs were
                                                 discovered, and the data

                         Digital Technical Journal Vol. 3 No. 2 Spring 1991  15

 


           FDDI Data Link Development



             link performance and ring        Conclusion
             stability exceeded the               The development of the
             expectations. The steady-           FDDI data link and the
             state testing was then              chip set represents a
             augmented by introducing            major accomplishment and
             pathological conditions             technical breakthrough in
             into the ring such as               the high-speed LAN area.
             stations with duplicate             Significant contributions
             addresses and noise on              were made by Digital
             the fiber. The promiscuous          in the area of FDDI MAC
             capture modes and frame             algorithms and protocols to
             /event time stamping proved         improve the performance and
             invaluable in analyzing             robustness of the FDDI LAN.
             subsequent behavior in              The FDDI data link chips
             these cases as the effects          described in this paper
             were often complex. Station         are used in all members
             management implementation           of Digital's FDDI product
             was highly stressed and             line, including bridges,
             performed without error.            wiring concentrators,

              Finally, the same testers          and adapters. These
             were configured as bridges,         products have benefited
             and prototype Ethernet-to-          tremendously from the
             FDDI bridging firmware was          verification and test
             introduced. At this stage           method adopted. Digital
             several minor deficiencies          has built on its knowledge
             with the control algorithms         and experience in systems,
             required for bridging               networks, computer-aided
             were detected. These                design/simulation, and
             deficiencies were due               semiconductors to provide
             mainly to insufficient              FDDI design, development,
             analysis of FDDI bridging           and methodology. Again,
             requirements by the test            Digital has shown industry
             team; therefore, the test           leadership by producing the
             cases were correspondingly          FDDI chip set and products.
             inadequate. All problems
             were repeatable on the           Acknowledgments
             simulation test bed,
             and bug fixes with new               The authors acknowledge the
             tests were developed.               technical contributions of
             No deficiency was severe            Raj Jain, Brian Myrick,
             enough to prevent testing           Chuck Lee, and K.K.
             the prototype FDDI bridges          Ramakrishnan. The authors
             and the development of more         would like to thank Jerry
             efficient algorithms for            Hutchison, Bill Cronin, and
             the FDDI bridge products.           Raj Jain for their review
                                                 and several useful comments
                                                 on this paper.







          16  Digital Technical Journal Vol. 3 No. 2 Spring 1991

 


                                                     FDDI Data Link Development



          References

             1. R. Jain,
                "Characteristics of                 Overview of the Common
                Fiber Distributed Data              Node Software," Digital
                Interface (FDDI),"                  Technical Journal, vol.
                IEEE Transactions on                3, no. 2 (Spring 1991,
                Communications, vol. 38,            this issue): 42-52.
                no. 8 (August 1990).             8. MAC Bridge

             2. R. Jain, "Performance               Specifications,
                Analysis of FDDI Token              ANSI/IEEE Standard
                Ring Networks: Entity               802.1d (unapproved
                Effect of Parameters and            draft), P802.1d
                Guidelines for Setting              /D8 (New York: The
                TTRT," Proceedings of               Institute of Electrical
                the ACM SIGCOMM '90                 and Electronic
                (September 1990).                   Engineers,Inc., 1989).

             3. Standard for the                 9. B. Stewart, W. Hawe, and
                Transmission of IP                  A. Kirby, "Local Area
                Datagrams Over FDDI                 Network Connection,"
                Networks, Internet                  Telecommunications
                Engineering Task Force              Magazine (April 1984).
                (IETF) RFC 1103 (June            10.H. Yang and K. K.
                1989).                              Ramakrishnan, "Frame

             4. M. J. Johnson, "Fairness            Content Independent
                of Channel Access                   Stripping for Token
                for Non-Time-Critical               Rings," Proceedings
                Traffic Using the FDDI              of the ACM SIGCOMM '90
                Token Ring Protocol,"               (September 1990).
                NASA Ames Research               11.K. K. Ramakrishnan
                Center, Research                    and H. Yang, "Frame
                Institute for Advance               Content Independent
                Computer Science, RIACS             Stripping for Token
                TR 86.9 (March 1986).               Rings," Proceedings of

             5. Token Ring Access                   the 15th Conference on
                Method and Physical                 Local Computer Networks
                Layer Specifications,               (October 1990).
                ANSI/IEEE Standard               12.Fiber Distributed Data
                802.5-1989 (New                     Interface (FDDI) Media
                York: The Institute                 Access Control, American
                of Electrical and                   National Standard, ANSI
                Electronic Engineers,               X3.139-1987 (June 1988).
                Inc., 1989).

             6. R. M. Grow, "A Timed             13.J. Hutchison, C.
                Token Protocol for                  Baldwin, and B.
                Local Area Networks,"               Thompson, "Development
                Electro/82, Token Access            of the FDDI Physical
                Protocols (17/3) (May               Layer," Digital
                1982).                              Technical Journal, vol.
             7. P. Ciarfella, D. Benson,            3, no. 2 (Spring 1991,
                and D. Sawyer, "An                  this issue): 19-30.

                         Digital Technical Journal Vol. 3 No. 2 Spring 1991  17
=============================================================================
Copyright 1991 Digital Equipment Corporation.  Forwarding and copying of this 
article is permitted for personal and educational purposes without fee 
provided that Digital Equipment Corporation's copyright is retained with the 
article and that the content is not modified. This article is not to be 
distributed for commercial advantage. Abstracting with credit of Digital 
Equipment Corporation's authorship is permitted.  All rights reserved.
=============================================================================
