Words: 5025
Industry Standard Architecture and Technology Rev. 9. 31 Student guide 1 of 2 Use of this material to deliver training without prior written permission from HP is prohibited. ? Copyright 2009 Hewlett-Packard Development Company, L. P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.
This is an HP copyrighted work that may not be reproduced without the written permission of HP. You may not use these materials to deliver training to any person outside of your organization without the written permission of HP. Intel, Intel Itanium, and the Intel Inside logo are trademarks of Intel Corporation in the U. S. and other countries. Java is a US trademark of Sun Microsystems, Inc. Microsoft, Windows, and Windows NT are U. S. registered trademarks of Microsoft Corporation. UNIX is a registered trademark of The Open Group.
Printed in US Industry Standard Architecture and Technology Student guide (1 of 2) September 2007, September 2009 Contents Module 1 — Introduction to Server Technologies Objectives ………………………………………………………………………………………… 1 Server design…………………………………………………………………………………….. 2 Customer environments …………………………………………………………………… 2 Small-business servers ………………………………………………………………. 3 Enterprise ervers…………………………………………………………………….. 3 Servers differentiated from desktops ……………………………………………… 3 Customer requirements……………………………………………………………………. 4 Server functions ……………………………………………………………………………. 5 Server chassis types……………………………………………………………………….. 6 Form factor …………………………………………………………………………………. Racks…………………………………………………………………………………………. 7 Server technologies……………………………………………………………………………… 8 System boards ……………………………………………………………………………… 8 Processors …………………………………………………………………………………… 9 Memory……………………………………………………………………………………… Bus………………………………………………………………………………………….. 10 Chipsets……………………………………………………………………………………. 10 Storage components…………………………………………………………………….. 10 Drive arrays……………………………………………………………………………….. 10 System clock ……………………………………………………………………………….. 12 Data Flow ………………………………………………………………………………….. 2 Power ……………………………………………………………………………………….. 13 Cooling …………………………………………………………………………………….. 14 Learning check ………………………………………………………………………………….. 15 Module 2 — Chipsets and Server Architecture Objectives ………………………………………………………………………………………… 1 Chipsets …………………………………………………………………………………………… Architecture ………………………………………………………………………………………. 2 Evolution of chipsets and server architecture……………………………………………….. 3 Server architectures and processor chipsets………………………………………………… 4 Transitioning from FSB to point-to-point links……………………………………………….. 5 Comparing FSB and point-to-point link-based architectures ……………………….. 5 Changing from Megahertz to Gigatransfers ……………………………………. Comparing traditional and non-uniform memory architecture …………………….. 6 Servers based on traditional Intel Xeon front-side bus technology………………… 8 Servers based on point-to-point link technology……………………………………… 9 Rev. 9. 31 i Industry Standard Architecture and Technology Xeon processors with Core or Netburst microarchitecture and server architecture … 10 Bandwidth………………………………………………………….. …………………….. 10 Memory latency…………………………………………………………………………… 1 Scalability ………………………………………………………………………………….. 1 1 Evolution of Intel microarchitectures and processors ……………………………….. 12 Introduction of Nehalem microarchitecture ………………………………………………… 12 Intel Nehalem Microarchitecture……………………………………………………….. 13 Intel Xeon processor 5500 series………………………………………………………. 13 Integrated memory controller……………………………………………………… 4 Intel QuickPath Technology ……………………………………………………….. 15 Three-level cache hierarchy ……………………………………………………….. 16 AMD Opteron processors and server architecture ……………………………………….. 17 Direct Connect Architecture …………………………………………………………….. 17 Integrated memory controller …………………………………………………………… 18 HyperTransport Technology …………………………………………………………….. 8 Primary design difference for single, dual, or multi-core systems ……………….. 19 Significance of the crossbar switch …………………………………………………… 19 Socket designs for dual-core Revision F processors………………………………… 20 Socket design for quad-core processors……………………………………………… 20 AMD split-plane technology……………………………………………………………. 21 Common chipsets ………………………………………………………………….. …………. 2 Intel 5520 and 5500 chipsets…………………………………………………………. 23 Intel 5000P chipset ……………………………………………………………………… 24 Intel 7300 Chipset……………………………………………………………………….. 25 Features and benefits ……………………………………………………………… 25 Intel 7400 chipset………………………………………………………………………… 26 Intel E7520 chipset………………………………………………………………………. 7 Processor subsystem ……………………………………………………………….. 28 I/O subsystem………………………………………………………………………. 28 Intel E8500 chipset ……………………………………………………………………… 29 North Bridge………………………………………………………………………… 29 XMB memory controller …………………………………………………………… 30 Partitioning for electrical isolation ………………………………………………. 0 AMD 8000 series chipset……………………………………………………………….. 31 Memory subsystem…………………………………………………………………. 32 I/O subsystem………………………………………………………………………. 32 Bandwidth …………………………………………………………………………… 32 64-bit architecture………………………………………………………………….. 33 Opteron 8400 Series processors……………………………………………………… 34
Summary………………………………………………………………………………………… 35 Learning check …………………………………………………………………………………. 36 ii Rev. 9. 31 Contents Module 3 — Processors and Multiprocessing Objectives ………………………………………………………………………………………… 1 Choosing processors……………………………………………………………………………. 2 How processors work …………………………………………………………………………… Processor components…………………………………………………………………….. 4 How processors handle instructions ……………………………………………………. 5 Clock synchronization …………………………………………………………………….. 5 Processor performance and clock cycles ……………………………………………………. 6 EPIC ………………………………………………………………………………………….. 6 Multiprocessor servers ………………………………………………………………………….. Sharing the workload …………………………………………………………………….. 7 Sharing memory and I/O resources …………………………………………………… 8 Other factors ……………………………………………………………………………….. 9 Mixing processors……………………………………………………………………………… 10 Processor steppings ……………………………………………………………………… 10 Intel support for processor mixing……………………………………………………… 1 Operating system support for processor mixing …………………………………….. 1 1 Logical processors and hyper-threading technology……………………………………… 12 Thread scheduling………………………………………………………………………… 12 64-bit extensions —AMD64 and Intel 64 …………………………………………………. 13 Instruction set and registers……………………………………………………………… 13 Memory addressability ………………………………………………………………….. 4 Operating modes…………………………………………………………………………. 14 Micro-architectural differences …………………………………………………………. 15 Comparing Intel and AMD processors ……………………………………………….. 15 Improvements to silicon process technology……………………………………………….. 16 A milestone…. x86 processors on 90nm technology……………………………….. 16 Dual-core processors …………………………………………………………………………… 6 HyperThreading differences for single and dual-core processors………………… 17 Multiple core processors………………………………………………………………………. 18 Reducing and controlling power use ……………………………………………………….. 18 NX, XD, and DEP for data protection ……………………………………………………… 19 Hardware assisted virtualization ……………………………………………………………. 20 AMD-V and VT virtualization technology ……………………………………………. 0 Processor ring layers…………………………………………………………………….. 21 Current processors in industry standard servers………………………………………….. 23 Identifying processors and platforms…………………………………………………. 23 Identifying AMD Opteron processors…………………………………………… 23 Identifying Intel Xeon Processors ………………………………………………… 23 Intel Xeon processors ……………………………………………………………………. 4 Xeon dual-core processors ……………………………………………………….. 24 Xeon quad-core processors ………………………………………………………. 25 Xeon six-core processors ………………………………………………………….. 28 The new Intel Nehalem microarchitecture ……………………………………… 29 Rev. 9. 31 iii Industry Standard Architecture and Technology AMD Opteron processors………………………………………………………………. 32 AMD Direct Connect I/O Architecture …………………………………………. 2 Integrated memory controller…………………………………………………….. 32 HyperTransport links……………………………………………………………….. 33 Interpreting model numbers for Opteron processors…………………………. 34 Similar design for single, dual, or multi-core processors ……………………. 35 Opteron dual-core processors……………………………………………………. 35 Quad-core Opteron processors………………………………………………….. 7 Six-core Opteron processors………………………………………………………. 41 Summary………………………………………………………………………………………… 44 Learning check …………………………………………………………………………………. 45 Module 4 — Memory and Cache Objectives ………………………………………………………………………………………… 1 Memory …………………………………………………………………………………………… Narrowing the performance gap……………………………………………………….. 2 DRAM……………………………………………………………………………………………… 3 DIMMs ………………………………………………………………………………………. 3 Basic DRAM operation …………………………………………………………………… 4 DRAM storage density and power consumption …………………………………….. 6 Improving memory performance………………………………………………………… Memory access time………………………………………………………………………. 7 Memory bus speed ……………………………………………………………………….. 7 Burst mode access…………………………………………………………………………. 7 System bus timing………………………………………………………………………….. 8 SDRAM technology……………………………………………………………………………… 9 Bank interleaving ………………………………………………………………………… 0 Increased bandwidth …………………………………………………………………….. 1 1 Registered SDRAM modules ……………………………………………………………. 1 1 DIMM configurations …………………………………………………………………….. 12 Single-sided and double-sided DIMMs …………………………………………. 12 Single-rank, dual-rank, and quad-rank DIMMs………………………………… 12 The significance of memory ranks ……………………………………………….. 3 Advanced memory technologies…………………………………………………………….. 14 Three generations of DDR SDRAM technologies ……………………………………. 14 Double Data Rate and pumped buses ………………………………………….. 15 DDR-1 SDRAM …………………………………………………………………………….. 16 DDR-2 SDRAM…………………………………………………………………………….. 17 DDR-3 SDRAM ……………………………………………………………………………. 8 Module naming conventions and peak bandwidth ………………………….. 19 Limitations of traditional DIMM architectures……………………………………….. 20 Fully-Buffered DIMMs ……………………………………………………………………. 21 Rambus DRAM …………………………………………………………………………… 22 iv Rev. 9. 31 Contents DIMM error detection and correction technologies……………………………………… 23 The increasing possibility of memory errors ………………………………………… 3 Types of memory errors …………………………………………………………………. 24 Parity checking …………………………………………………………………………… 25 Error checking and correcting memory………………………………………………. 26 Advanced ECC …………………………………………………………………………… 27 Summary of error protection with Advanced ECC …………………………… 28 Memory protection technologies……………………………………………………………. 9 Online spare memory …………………………………………………………………… 29 Special requirements for online spare memory……………………………….. 30 Mirrored memory …………………………………………………………………………. 31 Special requirements for mirrored memory …………………………………….. 31 Lockstep memory mode…………………………………………………………………. 32 Memory protection summary ………………………………………………………….. 2 Hot-plug mirrored memory …………………………………………………………….. 33 RAID memory …………………………………………………………………………….. 34 Comparing RAID memory and drive arrays…………………………………… 35 Cache……………………………………………………………………………………………. 35 Cache levels and placement …………………………………………………………… 36 Cache lines and tag RAM ……………………………………………………………… 6 Improving cache performance ………………………………………………………………. 37 Cache architectures ……………………………………………………………………… 37 Writing to cache …………………………………………………………………………. 37 Reading from cache …………………………………………………………………….. 38 Summary………………………………………………………………………………………… 39 Learning check …………………………………………………………………………………. 0 Module 5 — Bus Architecture Objectives ………………………………………………………………………………………… 1 Bus architecture overview………………………………………………………………………. 2 Address bus ………………………………………………………………………………… 3 Data bus …………………………………………………………………………………….. 3 Control signals……………………………………………………………………………… Bus performance ………………………………………………………………………………… 4 Bus speed …………………………………………………………………………………… 4 Bus cycles …………………………………………………………………………………… 5 Maximum transfer rate ……………………………………………………………………. 5 Increasing the performance through the maximum transfer rate…………….. 6 Bus mastering ………………………………………………………………………………. Bus arbitration ……………………………………………………………………………… 6 Rev. 9. 31 v Industry Standard Architecture and Technology PCI …………………………………………………………………………………………………. 7 Features of the PCI bus……………………………………………………………………. 8 Considerations for PCI configuration…………………………………………………… 9 PCI bus frequency arbitration……………………………………………………………. PCI bus number assignments ………………………………………………………….. 10 PCI hot plug support…………………………………………………………………….. 10 IEEE 1394……………………………………………………………………………………….. 10 PCI-X………………………………………………………………………………………………. 1 1 PCI-X features ……………………………………………………………………………… 12 PCI-X protocol enhancements ………………………………………………………….. 13 PCI-X register-to-register protocol…………………………………………………. 13 PCI-X hierarchical structure ……………………………………………………………… 15 PCI hot plug support in PCI-X …………………………………………………………… 15 Adapter card selection…………………………………………………………………… 16 Card slot support……………………………………………………………………. 6 Comparing buses and slots ……………………………………………………….. 17 Interoperability matrix ……………………………………………………………… 17 PCI-X 2. 0……………………………………………………………………………………. 18 Transitioning from the parallel bus system ………………………………………………… 19 PCI Express……………………………………………………………………………………… 0 Features ……………………………………………………………………………………. 20 PCIe technology………………………………………………………………………….. 21 PCIe architecture……………………………………………………………………. 21 Dual-simplex lanes …………………………………………………………………. 22 8b/10b encoding………………………………………………………………….. 22 Performance ………………………………………………………………………………. 3 PCIe data transfer rates …………………………………………………………… 23 Evolution of I/O bandwidth ……………………………………………………… 24 Backward compatibility …………………………………………………………… 24 Form factors ………………………………………………………………………………. 25 Card interoperability ……………………………………………………………………. 26 75-watt and 150-watt PCI Express slots ……………………………………………… 6 USB ………………………………………………………………………………………………. 27 USB connectors…………………………………………………………………………… 28 Series A ……………………………………………………………………………… 28 Series B ………………………………………………………………………………. 28 Emerging hardware support for I/O Virtualization ……………………………………… 29 Intel VT-d and AMD IOMMU ………………………………………………………….. 9 IOV – I/O Virtualization at the endpoint………………………………………. 30 Server-edge I/O virtualization…………………………………………………………. 30 Summary…………………………………………………………………………………………. 31 Learning check …………………………………………………………………………………. 32 vi Rev. 9. 31 Contents Module 6 — Server Storage Objectives ………………………………………………………………………………………… ATA/IDE…………………………………………………………………………………………… 2 Limitations of parallel ATA architecture ………………………………………………… 2 Serial ATA ………………………………………………………………………………………… 3 SATA enhancements ………………………………………………………………………. 3 LVD signaling …………………………………………………………………………. 8b/10b encoding……………………………………………………………………. 5 Reduced connector pin count ……………………………………………………… 5 SATA device connectors…………………………………………………………….. 6 Point-to-point connections…………………………………………………………… 6 Recommended uses for SATA technology ……………………………………….. 6 Serial ATA specification ………………………………………………………………….. 7 SATA 1. Gb/s (with extensions) and SATA 3. 0 Gb/s ……………………….. 7 Introduction to SCSI …………………………………………………………………………….. 8 SCSI-3 ……………………………………………………………………………………….. 9 Wide-Ultra2 SCSI-3 ……………………………………………………………….. 10 Ultra3 …………………………………………………………………………………. 1 1 Ultra320………………………………………………………………………………. 1 SCSI standards summary………………………………………………………………… 12 SCSI characteristics ………………………………………………………………………. 13 SCSI protocol compatibility …………………………………………………………….. 13 SCSI addressing…………………………………………………………………………… 14 Configuring SCSI IDs……………………………………………………………….. 15 SCSI device ID guidelines …………………………………………………………. 5 Logical unit numbers ……………………………………………………………….. 16 SCSI communication……………………………………………………………………… 17 SCSI bus phases …………………………………………………………………….. 17 SCSI bus transactions……………………………………………………………………. 19 Negotiation …………………………………………………………………………. 19 Disconnect and reconnect………………………………………………………… 0 Tagged command queuing ………………………………………………………. 20 Electrical signaling systems ………………………………………………………. 21 Cables……………………………………………………………………………………… 23 SCSI connectors ………………………………………………………………………….. 24 SCSI troubleshooting ……………………………………………………………………. 27 SCSI compared to parallel ATA/IDE hard drives…………………………………… 8 Limitations of SCSI architecture……………………………………………………….. . 29 Rev. 9. 31 vii Industry Standard Architecture and Technology Comparing parallel and serial SCSI ……………………………………………………….. 30 Serial Attached SCSI …………………………………………………………………………… 31 SAS performance …………………………………………………………………………. 31 SAS protocol evolution ………………………………………………………………….. 2 SAS-1 …………………………………………………………………………………. 32 SAS-2…………………………………………………………………………………. 32 SAS-2. 1……………………………………………………………………………….. 32 Features and use …………………………………………………………………………. 33 SAS/SATA interoperability …………………………………………………………….. 34 SAS devices ………………………………………………………………………………. 5 Initiators ……………………………………………………………………………… 35 Expanders …………………………………………………………………………… 35 Targets ……………………………………………………………………………….. 35 Differential signaling ……………………………………………………………………. 37 Cabling and connectors………………………………………………………………… 38 SAS connectors …………………………………………………………………….. 8 Mini SAS 4x cable connectors and receptacles ……………………………… 38 Mini SAS 8x cable connectors…………………………………………………… 40 SAS topologies ……………………………………………………………………………. 41 Zoning……………………………………………………………………………………… 44 SAS migration ……………………………………………………………………………. 44 Backplane design ……………………………………………………………………….. 5 Cabling and hot swap ………………………………………………………………….. 46 Management and fault isolation………………………………………………………. 46 SAS advantages …………………………………………………………………………. 46 Serial SCSI cables …………………………………………………………………. 47 SAS and Fibre Channel compatibility ……………………………………………………… 48 Fibre Channel ………………………………………………………………………………….. 8 SATA, SAS, and Fibre Channel comparison ……………………………………………… 49 iSCSI……………………………………………………………………………………………… 49 How iSCSI works ………………………………………………………………………… 50 Emergence of solid state drives for servers ………………………………………………… 51 Performance of solid state drives ………………………………………………………. 51 Reliability and operational environment of solid state drives……………………… 1 Summary………………………………………………………………………………………… 52 Learning check …………………………………………………………………………………. 53 viii Rev. 9. 31 Contents Module 7 — Array Technology Objectives ………………………………………………………………………………………… 1 Drive array concepts ……………………………………………………………………………. 2 Logical drives……………………………………………………………………………….. Array features and advantages ………………………………………………………………. 4 HP implementation for highly reliable array control…………………………………. 5 Tagged command queuing (TCQ) ……………………………………………………… 5 Number of disks in an array ………………………………………………………………….. 5 RAID technology overview …………………………………………………………………….. 6 JBOD…………………………………………………………………………………………. Software and hardware RAID implementations ……………………………………… 7 Software-based RAID ……………………………………………………………….. 7 Hardware-based RAID ……………………………………………………………… 8 Mean time between failure………………………………………………………………. 9 Extending MTBF ……………………………………………………………………… 9 Striping factor…………………………………………………………………………….. 0 RAID levels………………………………………………………………………………… 10 RAID levels supported by HP RAID controllers ……………………………………………. 1 1 RAID 0 — Disk striping ………………………………………………………………….. 1 1 RAID 1 — Disk mirroring ……………………………………………………………….. 12 RAID 1+0 — Striped mirror sets……………………………………………………….. 13 RAID 5 — Distributed data guarding …………………………………………………. 4 RAID 5 performance ……………………………………………………………….. 15 RAID 6 — Advanced data guarding …………………………………………………. 16 HP implementation of RAID 6 …………………………………………………….. 16 RAID 6 performance ……………………………………………………………….. 17 When to use RAID 6 ……………………………………………………………….. 17 RAID fault-tolerance comparison ………………………………………………………. 8 RAID level performance comparison ………………………………………………………. 19 RAID failed drive recovery time ……………………………………………………….. 20 Optimizing the stripe size………………………………………………………………. 21 Summary………………………………………………………………………………………… 22 Learning check …………………………………………………………………………………. 23 Rev. 9. 31 ix Industry Standard Architecture and Technology
Module 8 — Advanced Storage Technology Objectives ………………………………………………………………………………………… 1 Advanced storage solutions …………………………………………………………………… 2 Storage types…………………………………………………………………………………….. 2 Primary storage ……………………………………………………………………………. 2 Internal storage ………………………………………………………………………. External storage ……………………………………………………………………… 3 Secondary storage systems………………………………………………………………. 4 Storage implementations ………………………………………………………………………. 5 Comparing key features of implementations………………………………………….. 5 Direct-attached storage …………………………………………………………………… 6 Network-attached storage ……………………………………………………………….. Storage Area Networks ………………………………………………………………….. 8 Comparison of NAS and SANs …………………………………………………… 9 Types of SANs …………………………………………………………………………………. 10 Homogeneous operating system SAN ……………………………………………….. 10 Heterogeneous operating system SAN ………………………………………………. 10 Open SANs ……………………………………………………………………………….. 1 Fibre Channel-based SANs …………………………………………………………….. 12 Current topologies ………………………………………………………………….. 12 Ports……………………………………………………………………………………. 13 Addressing …………………………………………………………………………… 13 IP-based SAN ……………………………………………………………………………… 14 iSCSI…………………………………………………………………………………… 4 SAN topologies…………………………………………………………………………………. 15 Cascaded Fabrics ………………………………………………………………………… 16 Meshed Fabrics …………………………………………………………………………… 17 Ring Fabric…………………………………………………………………………………. 18 Backbone SAN Fabric ………………………………………………………………….. 19 SAN-Based High Availability ………………………………………………………….. 0 Virtual SANs ……………………………………………………………………………………. 20 Summary………………………………………………………………………………………… 20 Learning check …………………………………………………………………………………. 21 x Rev. 9. 31 Introduction to Server Technologies Module 1 Objectives An understanding of how subsystems work together within server architecture is an important foundation for integration, service, and support professionals.
This knowledge contributes to your ability to select and configure a server to meet specific application needs, provide the best performance, and troubleshoot systems and components. This module introduces server technologies that are explained in more detail during the remainder of the course. After completing this module, you should be able to: ? Describe what differentiates servers from desktops and list some of their applications Briefly explain the function of the following server components and subsystems: ? ? ? ? ? ? ? ? ? ? System boards Processors Memory Bus Chipsets Storage Power Racks System clock 9. 31 1–1
Industry Standard Architecture and Technology Server design Building a server requires integrating system components into an architecture that meets specific server design requirements. Each server is designed with a set of subsystems to create a balanced, high-performance, and cost-effective computer architecture. Server architecture is tested throughout the design and manufacturing process. Some of the key components found in industry standard servers include system boards, processors, chipsets, buses (I/O interconnects), memory, storage components, power, and mechanical components (for example, server chassis).
Servers can run many programs simultaneously and process requests for data from many users at once. To ensure performance, the system architecture in a server is designed to eliminate or minimize bottlenecks that affect the speed at which information is accessed or processed. The system architecture also provides high availability, networking, and management features. As high-performance computers that are designed to provide data to users on a network, servers require: ? ? ? One or more processors A large amount of memory and internal or external storage Fast input/output (I/O) subsystems
Manufacturers strive to enhance the quality, reliability, maintenance, and performance of servers while reducing costs. Factors considered include: ? ? ? ? Customer environments and profile Server function Customer requirements New technologies and emerging standards Customer environments Servers typically: ? ? ? Support from one to hundreds or thousands of clients Run 24 hours a day, 7 days a week Provide shared resources and services to all users, such as data, printers, modems, applications (such as email), and Internet connectivity Run many applications simultaneously Do not require high-power graphics ? Servers are available in all sizes, speeds, and powers, ranging from one to eight processors. Additionally, servers are available in a variety of form factors, from blades to rack-mountable thin servers to stand-alone towers. 1–2 9. 31 Introduction to Server Technologies A wide range of industry-standard servers results in the availability of models that are appropriate for small, medium, and enterprise businesses. A distinction is often made in the type of customer environments for which servers are designed.
Features are built into servers that are most suitable for specific uses. A general-purpose server is built on a well-rounded platform. It can be used in any environment, from small to large businesses, for purposes ranging from departmental use to clustered applications in the enterprise. Small-business servers Because small businesses usually do not have extensive experience using server systems, a small-business server has features that increase its ease of use and deployment.
This type of server handles file, print, and client/server application requirements with a broad set of features designed to manage all typical server tasks in a limited environment. Quick recovery is required because downtime immediately impacts the ability of a small office to conduct business. A small-business server can function as a client workstation and simultaneously perform a normal role as a server. This dual usage imposes additional requirements for power management and configuration. Enterprise servers
The enterprise server can be used in any environment but is frequently a building block in a large group of servers where each server performs a special-purpose task such as: ? ? ? Handling and routing email Storing financial data Administering a database Because this server is an indispensable part of the organization, it must be highly available. Therefore, software and hardware components must be in place to eliminate unplanned downtime. Servers differentiated from desktops Servers can be differentiated from desktop computers based on hardware and technology.
Although some customers use high-end desktop computers to perform traditional server functions, a desktop computer does not provide the necessary reliability and availability to ensure business continuity. To improve reliability and availability, many servers include: ? Hot-plug hard drives and memory — Allow drive or memory removal and replacement or upgrade without shutting down the server Redundant and hot-plug fans — Provide advanced cooling systems, and allow fan removal and replacement without shutting down the server ? 9. 31 1–3 Industry Standard Architecture and Technology ?
Redundant and hot-plug power supplies — Eliminate the power supply as a single point of failure, and allow power supply removal and replacement without shutting down the server Hot-plug PCI slots — Allow PCI card replacement without shutting down the server Management tools — Enable local and remote server management Multiple processors — Servers support multiple processors, whereas desktop computers support only one processor. Larger amounts of memory — Servers support significantly larger amounts of memory than desktop computers. More internal and external storage — Servers are designed for more internal and external storage.
Desktop computers are not designed for large amounts of either type of storage. System architecture designed to minimize bottlenecks — Servers are designed to minimize bottlenecks that can be encountered when multiple users access a server simultaneously. Reduced video and sound capabilities — Desktop computers typically contain more advanced video and sound technologies than servers because they are more often used for running and viewing graphic-intensive games and programs. ? ? Other characteristics of server design include: ? ? ? ? ? Customer requirements Business computing requires servers that perform a variety of functions.
Engineers work closely with customers and software partners to design, integrate, and test servers. This collaboration produces a variety of servers and solutions that best address business computing needs. Servers are designed for specific markets, needs, and applications where network security is important and large amounts of data must be stored and processed. Most importantly, servers offer maximum uptime and reliability to ensure that customers’ businesses stay up and running. 1–4 9. 31 Introduction to Server Technologies Server functions Some server functions include: ? ?
Application server — Provides client access to applications Back-end server — Holds at least one database to which front-end servers connect when relaying requests from clients Bridge server — Filters and forwards network traffic Database server — Stores, retrieves, and manages data Domain Name Server (DNS) — Provides resolution from hostnames to IP addresses and so forth Dynamic Host Configuration Protocol (DHCP) server — Assigns IP addresses to clients on the network Fax server — Manages, sends, and receives faxes from client workstations File server — Stores files for clients Firewall server — Monitors incoming and outgoing network traffic and prevents unauthorized access File transfer protocol (FTP) server — Enables downloading and uploading of files Front-end server — Receives requests from clients and relays them to the appropriate back-end server Gateway server — Connects dissimilar networks Mail server — Manages incoming and outgoing mail Print server — Manages print jobs from clients Proxy server — Filters outgoing network requests Remote Access Server (RAS) — Allows users to remotely access networks Router server — Manages traffic between networks Simple Mail Transfer Protocol (SMTP) server — Routes email messages through an organization or the Internet to their final destination Web server — Delivers web pages Windows Internet Name Service (WINS) server — Provides network name resolution ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 9. 31 1–5 Industry Standard Architecture and Technology Server chassis types Servers are available in three basic chassis types, which are generally designed to best serve an particular purpose and environment. ? Blade servers are modularized ultra-dense servers designed for power efficiency. These servers provide high performance and availability for multi-tiered environments, centralized redundant power, and enhanced management capabilities Tower servers usually offer the greatest capacity for internal expansion. Rack-mounted servers are usually intended for data center and external storage environments. ? ? Form factor
In the computing industry the form factor describes the physical dimensions of systems. It also describes major subsystem components, such as drives and system boards. Standardized form factors enable the interchangeable use of hardware from different vendors and product generations. The form factor is especially important when planning or performing an installation in a rack. The dimensions of the server must be considered for efficient use of space. A rack is measured in units. One unit (1U) equals 1. 75 inches (4. 4 centimeters). A standard rack measures 42U in height. A server that is to be installed in a rack has a unit measurement that specifies its height, such as 1U or 7U. 1–6 9. 31
Introduction to Server Technologies Racks A rack is a metal frame or cabinet into which computer components can be mounted. Racks typically ship in standard widths of 19 or 23 inches; height is specified by a unit (U) of measure that is 1. 75 inches from top to bottom. Racks ship in a variety of sizes, from 14U to 47U, to offer a range of rackmount capacity. Common features include ventilated doors and side panels, sliding shelves, and locks. Some racks provide a water-cooled environment. Manufacturers also offer a variety of options, including a selection of rack-mountable monitors and keyboards, console switches, and cable-management solutions.
In some cases online tools can help customers design and configure a rack solution to best fit their space, capacity, power, and cooling requirements. 9. 31 1–7 Industry Standard Architecture and Technology Server technologies This section provides a brief introduction to server subsystems and server technology, which are described in more detail in the following course modules. System boards A server system board, as shown in the following graphic, is an integrated circuit board containing most of the processing components. System boards come in three types: active, passive, and modular. The active board is the traditional system board (motherboard) that has embedded, active chipsets. The passive system board does not contain embedded chipsets.
Instead, it functions like an interconnect board, linking several integrated circuit component boards together. A modular system board is a hybrid between the active and passive types. It contains some embedded chipsets and acts as an interconnect board for upgradeable, integrated circuit component boards. Both active and modular system boards are used in current servers. The active board design offers the most cost-effective solution for entry-level servers and densityoptimized servers. The modular board design provides maximum scalability and component upgrades in a server. System board 1–8 9. 31 Introduction to Server Technologies Processors The processor functions as the brain of the server, controlling all activity among the server components.
A processor is composed of several components, located on the same integrated circuit and working together. Servers are manufactured with processors from a number of manufacturers. This training describes Intel and AMD processors. Engineers work closely with manufacturers of processors to ensure compatibility between the processor and the other server components. Typical system processor Memory Memory stores data needed by other components. It is used in several areas of the server, including the main system, cache, video, and drives. Memory is sometimes called RAM (random access memory). The basic types of RAM include dynamic ram (DRAM) and static RAM (SRAM).
DRAM is the most common memory type found in server memory subsystems. This memory technology is inexpensive but very slow compared to processor requirements. SRAM is used most exclusively in cache implementations. This memory technology is extremely fast, but is more expensive than DRAM and generates significant heat. Typical DRAM memory 9. 31 1–9 Industry Standard Architecture and Technology Bus A bus is a set of electrical circuits used to transport binary data that has been converted to electrical signals, enabling communication among all of the components of a server. Many types of buses work together in a server to provide high performance with minimal bottlenecks.
The basic types of buses found in a server include the following: ? ? Processor bus — Is internal to the processor Backside bus — Connects the processor or processors to the on-board cache memory Frontside bus — Connects the processor to the system controller chipset Memory bus — Connects the main memory to the memory controller Local I/O bus — Connects high-speed internal and external I/O devices to the I/O controller hub Expansion I/O bus — Connects I/O ports, expansion I/O controllers, and expansion slots to the I/O controller hub ? ? ? ? Each server bus type is made up of two electrical lines, which are also known as buses. The first type is the address or control bus.
This bus identifies the desired location within a target device where data might reside. It also carries control signals that indicate the purpose of the data transfer, such as whether a device is supposed to read or write the data. The second type is the data bus. Data moves between any two devices over the data bus. The data can be instructions for the microprocessor or information the microprocessor is transmitting. This information can pass to or from the memory or I/O subsystem. Chipsets The chipset is the central nervous system of the server. It controls communications and data exchange between all server components and determines the overall functionality of the server.
A chipset is a set of application-specific integrated circuits (ASICs) that has many basic server-logic functions integrated into one or more chips. Functions integrated into the ASICs include the system controller, the memory controller, the I/O controller, and the program interrupt controller. 1 – 10 9. 31 Introduction to Server Technologies Storage components Disk drives are a relatively fast, reliable, and economic way to store and access data, applications, and operating systems. Server storage typically consists of removable storage and permanent storage. Removable storage includes CD-ROM and DVD drives, disk drives, and tape backup drives.
Permanent storage includes IDE ATA, SCSI, SATA, and SAS disk drives. Drive arrays An array, sometimes called a RAID array, is a set of disks that acts like a large single disk, thereby providing higher performance and data fault tolerance than a multidisk system. Because storage needs have expanded beyond the server and the types of storage devices have increased, the need for a high-speed connection between servers and storage devices has become critical. Storage solutions designed to meet these needs include the following: ? Direct-attached storage (DAS) — Storage devices directly, internally, or externally connected to the server, such as disk drives, tape drives, and external storage systems.
Storage area network (SAN) — A dedicated, centrally managed, secure information infrastructure network that enables direct physical access to common storage devices or a storage pool. Network attached storage (NAS) — Dedicated, self-contained, intelligent servers that attach directly to the existing LAN. Data is transferred to and from clients over industry-standard network protocols using industry-standard file-sharing protocols. ? ? 9. 31 1 – 11 Industry Standard Architecture and Technology System clock A system clock coordinates the activity of the server components. The clock is actually a crystal. When electrical current is applied to the crystal, it vibrates at a constant rate.
The number of vibrations is measured in megahertz (MHz) or gigahertz (GHz). Each vibration is known as a clock cycle or clock tick. The clock cycles act as a sort of drumbeat that drives the work of all other components. Every action a component makes takes place on a clock tick. Some components work faster than others, getting their work done in fewer clock cycles. The challenge in server design is to connect all the components in a way that provides a constant data flow through the system but minimizes bottlenecks. The design of the server, combined with the user configuration of the server, determines whether a subsystem works to enhance performance or works as a bottleneck to restrict overall performance.
Data flow Data flow in a server can be viewed in a hierarchical, fastest-to-slowest, logical block diagram. Data must move through the system effectively from fast to slow and back. To keep the faster subsystems (processors and memory) working efficiently, the system depends on controllers, memory buffers, parallel buses, and concurrent read and write processes. 1 – 12 9. 31 Introduction to Server Technologies Power Server components require a tightly controlled and regulated supply of power. The power supply must be adequate to support the electrical load of the server in its standard configuration and support any added or upgraded components.
The server power components are a critical part of server fault prevention and fault tolerance and should be able to (1) predict and avoid failure, (2) allow preventive maintenance before failure, and (3) keep the server running in the event of component failure. Servers can include power-system designs that provide reliability through fault prevention and fault-tolerant features, including intelligent power supplies and fans, and redundant power supplies and fans. A typical power supply is shown in the following graphic. The power subsystem includes power, thermal, and airflow components. Power components can be classified by whether they are internal or external to the server.
Internal power components include the power supply, the voltage regulator module (VRM) (sometimes called the processor power module, or PPM), fans, and system board thermistors. External power components include uninterruptible power supply (UPS) and cables, the power-distribution unit, and the redundant A/C power source. Typical server power supply 9. 31 1 – 13 Industry Standard Architecture and Technology Cooling System cooling technology and components are critical to server operation. Redundancy and hot-swap capability for power and fans are important to preventing downtime. Thermal management and heat dissipation are issues that engineers need to resolve when designing servers. Current network servers tend to be densely packed and have multiprocessors that run at high speeds and generate heat.
Conventional cooling uses air convection, heat sinks, and fans or some combination of these elements. Shrouds and ducts might also be used. For example, a cooling shroud might be used to cover the processor and system battery and to direct airflow to the expansion cards and system memory. ? A passive cooling system utilizes heat sinks and natural convection. Heat sinks are blocks of metal that absorb heat and have fins or ridges to dissipate the heat. An active cooling system adds mechanical means. Fans are added to blow cooling air across or through the heat sink and other interior parts. ? Small to medium-sized servers generally use multiple fans, often associating a fan with each power supply.
Separate fans ensure the air movement through specific electronic areas. Small local fans sometimes increase the air velocity through power dense areas. Combining a closely-coupled local fan and heat sink produces a known air flow through the fins. Air exits the server’s interior after cooling the heat sink. Note Be sure to verify that you are installing or replacing optional or redundant fans in the correct locations as specified by the server’s documentation. Other cooling methods, such as heat pipes, thermoelectric, or water cooling, are possible. Water cooling can extract more heat than parts cooled by fins or ridges. 1 – 14 9. 31 Introduction to Server Technologies Learning check 1. 2.
Which type of memory is most commonly found in server memory subsystems? ………………………………………………………………………………………………….. What is the function of the address or control bus? ………………………………………………………………………………………………….. ………………………………………………………………………………………………….. 3. What is a dedicated, centrally managed, secure information infrastructure that enables direct and physical access to common storage devices or a storage pool known as? a. b. c. d. 4.
Drive array Direct-attached storage Storage area network Network attached storage Which server component coordinates the activity of the server components and regulates server data flow? a. b. c. d. Memory Processor System clock System bus 9. 31 1 – 15 Industry Standard Architecture and Technology 1 – 16 9. 31 Chipsets and Server Architecture Module 2 Objectives This module provides an overview of server chipsets and server architecture innovations. After studying this module, you should be able to: ? ? ? Define the term chipset and explain the role of the chipset. Explain the difference between instruction set architecture and micro-architecture.
Describe the crossbar switch and its role in reducing bottlenecks at the memory and I/O controllers. Describe the main features of the Intel Xeon and the AMD Opteron chipsets. Describe the basic differences between the traditional Intel Xeon and the AMD Opteron chipsets and the server architectures supported by each. Describe the Intel Nehalem microarchitecture implemented in the newest generation of Intel Xeon processors. Describe the architectural elements shared by the AMD Opteron and the new Intel Xeon processor technologies. Identify commonly used current chipsets and the basic features these chipsets support for servers. ? ? ? ? ? 9. 31 2–1 Industry Standard Architecture and Technology Chipsets
A chipset is a collection of integrated circuits, or chips, located on a server motherboard. The chipset is designed to work together to control the features and the functions of the motherboard. It determines how much memory can be installed, which processors can be used, and which types of interfaces the computer can support. The chipset provides a foundation to the overall architecture of a server. To eliminate bottlenecks and increase performance, the chipset must provide for an optimized and balanced architecture. Architecture As a computer engineering term, architecture is often applied broadly, but it consists of the following specific areas: ?
Instruction set architecture is a high-level abstraction of a computing system and its functionality as seen by a machine language programmer. It includes the instruction set, registers, word size, memory address modes, and address and data formats. Micro-architecture is a lower level description of how the system is organized. It provides more concrete details. It describes how the parts of the system connect and interoperate to implement the instruction set architecture. For example, the size of cache is a micro-architecture issue, but not an instruction set issue. System design includes all of the hardware components of the system, such as memory control