What is an Intelligent Storage System

Intelligent Storage Systems are RAID arrays that are:

•Highly optimized for I/O processing

•Have large amounts of cache for improving I/O performance

•Have operating environments that provide:

–Intelligence for managing cache

–Array resource allocation

–Connectivity for heterogeneous hosts

–Advanced array based local and remote replication options

Logical Components of the Host: LVM

• Responsible for creating and controlling host level logical storage

– Physical view of storage is converted to a logical view by mapping

– Logical data blocks are mapped to physical data blocks

• Usually offered as part of the operating system or as third party host software

• LVM Components:

– Physical Volumes

– Volume Groups

– Logical Volumes

NRU Page Replcement Algorithm (Enhanced Second-Chance Algorithm)

The second-chance algorithm can be enhanced by considering both the reference bit and the modify bit (refer second chance algorithm for information about R and M bit).

R and M bits can be used to build a simple algorithm.

When a process is started up, both page bits for all its pages are set to 0 by the operating system.

Periodically (e.g. on each clock interrupt), the R bit is cleared to distinguish pages that have not been referenced recently from those that have been referenced.

When a page fault occurs, the operating system inspects all the pages divides them into four categories (classes) based on the current values of their R and M bits.

	R M
Class 0	(0,0)	Not referenced (not recently used) Not modified (clean)	Best page to replace
Class 1	(0,1)	Not referenced, modified	Not as good because the page will need to be written out before replacement.
Class 2	(1,0)	Referenced, not modified	Probably will be used again soon
Class 3	(1,1)	Referenced, modified	Probably will be used again and will need to be written out before replacement.

When page replacement is called for, each page is one of these four classes. We examine the class to which that page belongs.

Although class 1 pages seem impossible, but they occur when a class 3 page has it its R bit cleared by a clock interrupts.

Clock interrupt do not clear the M bit because this information is needed to know

Whether the page has be written to disk or not.

We replace the first page/random page encountered in the lowest nonempty class.

It is better to remove a modified page that has not been referenced in at least one clock tick (typically 20msec) than a clean page that is in heavy use.

The main attraction of NRU is that is easy to understand, efficient to implement, and gives a performance that while certainly not optimal is often adequate.

This algorithm is used in the Macintosh, virtual-memory-management scheme. The major difference between this algorithm and the simpler clock algorithm is that here we give preference to those pages that have been modified to reduce the number of i/o s required.

Paging from Memory Management

One possible solution to the external fragmentation problem is to permit logical address space of a process to be noncontiguous. This allows to allocate physical memory to a processes wherever it is available.

One way of implementing this solution is through the user of a paging scheme.

The fragmentation problem with main memory are also common with backing store. Paging avoids the considerable problem of fitting the varying-resized memory portion onto the backing store.

When some code fragments or data residing in main memory to be swapped out, space must be found on the backing store.

Because of its advantages over the previous methods, paging in various form is commonly used in many operating system.

Components of Storage System Environment

Upon completion of this lesson, you will be able to:

•Describe the three components of storage system environment

–Host, Connectivity and Storage

•Detail Host physical and logical components

•Describe interface protocol

–PCI, IDE/ATA and SCSI

•Describe storage options

–Tape, optical and disk drives

Disk Drive Implimentation

Upon completion of this lesson, you will be able to :

• List and discuss various disk drive components

– Platter, spindle, read/write head and actuator arm assembly

• Discuss disk drive geometry

• Describe CHS and LBA addressing scheme

• Disk drive performance

–Seek time, rotational latency and transfer rate

• Law’s governing disk drive performance Enterprise flash drive

REST Guiding principles of the interface

The uniform interface that any REST interface must provide is considered fundamental to the design of any REST service.

• Identification of resources 

Individual resources are identified in requests, for example using URIs in web-based REST systems. The resources themselves are conceptually separate from the representations that are returned to the client. For example, the server does not send its database, but rather, perhaps, some HTML, XML or JSON that represents some database records expressed, for instance, in Finnish and encoded in UTF-8, depending on the details of the request and the server implementation.

• Manipulation of resources through these representations

When a client holds a representation of a resource, including any metadata attached, it has enough information to modify or delete the resource on the server, provided it has permission to do so.

REST Constraints

The REST architectural style describes the following six constraints applied to the architecture, while leaving the implementation of the individual components free to design:

Client–server: Clients are separated from servers by a uniform interface. This separation of concerns means that, for example, clients are not concerned with data storage, which remains internal to each server, so that the portability of client code is improved. Servers are not concerned with the user interface or user state, so that servers can be simpler and more scalable. Servers and clients may also be replaced and developed independently, as long as the interface is not altered.

Stateless: The client–server communication is further constrained by no client context being stored on the server between requests. Each request from any client contains all of the information necessary to service the request, and any session state is held in the client. The server can be stateful; this constraint merely requires that server-side state be addressable by URL as a resource. This not only makes servers more visible for monitoring, but also makes them more reliable in the face of partial network failures as well as further enhancing their scalability.

Memory Compaction / Compaction

To place all free memory together in one large block, the memory content is shuffled.

 

3 holes of size 100k,300k and 260k can be compacted into one hole of size 560k.

Memory Management from Page Replacement

In the presentation so far, the page fault rate is not a serious problem, because each page is faulted for at most one when it is first referenced. This representation is not strictly accurate.

Consider that, if a process of 10 pages actually uses only one half of them, than demand paging saves the I/O necessary t load the five pages that are nevred used.

We could also increase our degree of multiprogramming by running twice as many processes thus, if we had 40 frames, we could run eight processes, rather than the four that could run if each required 10 frames (five of which were never used).

If we increase our degree of multiprogramming, we are over-allocating memory. if we run six processes each of which is 10 pages in size, but actually uses only five pages, we have higher CPU utilization and throughput, with 10 frames to spare.

Optimal Algorithm

An optimal page-replacement algorithm has the lowest page-fault rate of all algorithms.

An optimal page-replacement algorithm also called OPT or MIN.

It is simply. Replace the page that will not be used for the longest period of time.

Use of this page-replacement algorithm guarantees that the lowest possible page fault rate for a fixed number of frames.

For example

We can create a FIFO queue to hold all pages in memory we replace the page at the head of the queue.

FIFO Algorithm

The simplest page replacement algorithm is a FIFO algorithm.

A FIFO replacement algorithm associate the time with each page when that page was brought into memory.

When a page must be replaced, the oldest page is chosen, it not strictly necessary to record the time when a page is brought in.

We can create a FIFO queue to hold all pages in memory. We replace the page at the head of the queue.

When a page is brought into memory, we insert it at the tail of the queue.

For our example reference string, out three frames are initially empty.

Page Replacement Algorithms

There are many different page replacement algorithms. Probably every operating system has its own unique replacement scheme.

How do we select a particular replacement algorithm? In genrate, want the one with the lowest page fault rate.

We  evaluate an algorithm by running it on a particular string of memory references and computing the number of page faults. The string of memory references is called reference string.

We can generate reference string:

1.       Artificially(by a random number generator for example) or

2.       By tracing a given system and recording the address of each memory reference.

Structure of the Page Table in Memory Management

Each operating system has its own methods for storing page tables.

Most of them allocate a page table for each process. A pointer to the page table is stored with the other register values in the process control block (PCB)

Hardware Support

The hardware implementation of the page table can be done in a number of different ways.

Way – 1

Memory Management Algorithms

1. First-fit:

          It is simplest algorithm.

          Memory manager scans list of holes and allocate first hole that is big enough.

Searching always starts at beginning of set of holes and stop as soon as it finds a free hole this is large enough to satisfy memory requirement of current process.

Hole is broken into 2 parts. One for process and one create a hole except it is exact fit.

Advantage: it is fast because it searches as little as possible.

2. Next-fit:

It is a minor variation in first-fit.