Please enable JavaScript.
Coggle requires JavaScript to display documents.
3. STORAGE AND MEMORY (FILE SYSTEMS (FILE SYSTEM COMPONENTS:
File system…
3. STORAGE AND MEMORY
FILE SYSTEMS
- FILE SYSTEM COMPONENTS:
File system defined by rules: heirarchical logical structure
- Disk Management - Organises disk blocks into names/files/folders (abstraction from hardware/logical storage) not tracks/sector no's
- Protection - keeps information secure (permissions)
- Reliability - protects loss / crashes
-
- FILE SYSTEM FORMATS
- FAT – File Allocation Table (old) Windows
- NTFS – New Technology File System Windows
- APFS – Apple File System MacOS
- Ext4 – Extended File System GNU/Linux
- And many others (XFS, JFS, …)
- BACKUPS
- manual - HDD, USB, SD card
- automatic - handled by OS e.g. to a partition or separate hard drive
- cloud - automatically syncs local data with online storage Dropbox, OneDrive, Google Drive
- (Distributed) Version control - saving incremental versions of data
PARTITIONING
- HARD DRIVES
- Primary Partition: Every HDD - one initial primary partition
- Active Partition: The partition on a hard disk drive where boot files are located
- Extended Partition: A partition on the disk drive that can be divided into multiple logical drives
- Logical Drive: A defined portion of storage on an extended partition
- Partition Tables
Allows the hard disk to be divided up into partitions
- MBR (Master Boot Record)
- It is the first sector on every disk.
- The location: track (cylinder) 0, side (head) 0, and sector 1
- Contains information on how logical partitions containing file systems are organised.
- Contains executable code (Boot Loader) acting as a loader for the installed OS.
- 3 Partitions + 1 Extended Partition ( with up to 23 logical partitions). Total 26 (DOS limitation) when alternate OS could have more.
- Limited to 2TB
- GPT (GUID Partition Table)
- Supports 128 partitions (without extended partition)
- Supports Zettabyte hard disks
- Required for booting
- UEFI (Unified Extensibel FIrmware Interface)
- 64bit hardware
PROCESSES
- Process v Program
A process is a program in execution - Process more than code; includes:
- Current activity
- Content of Processor’s registers
- Content of Memory
- Process Stack includes method parameters, return address and local variables
- Data section contains global variables
- Executable code of application
- Modern OS can run multiple concurrent prosses
- Pseudo parallelism - more processes running in parallel than Processors (multiprogramming or multiprocessing). Gives programs virtual processors. Applications think they have their own CPU and memory
- Process States:
- New: Process has been created
- Ready: Process is ready for execution/ assigned to CPU
- Running: Process is currently being executed
- Waiting: Process is waiting to continue execution on CPU
- Blocked: Process is blocked, e.g., due to waiting for I/O or an interrupt
- Terminated: Process has finished execution
MEMORY MANAGEMENT
- Why memory management?
- So multiple processes can use memory
- Allocate free memory on request
- Manage unused memory
- Virtual memory
- OS uses some secondary storage like volatile RAM (when actual RAM is running low).
- Latency (the delay in data tranfer following instruction)is increased.
- Using virtual memory slows PC - copying to a hard disk takes much longer than reading and writing RAM.
- Paging:
program/data divided in to pages few kB
- Swapping: The OS moves data from processes not immediately needed from RAM into virtual memoryand back into RAM when needed.
- Thrashing: - Process is thrashing if spends more time paging than executing.
If process needs pages in memory, OS loads them. If memory space insufficient, then needs to swap out pages. Later these pages are needed, then other pages swapped out...
COMMAND LINE INTERFACE
- TAB key - to autocomplete file/folder names
- UP key - cycle through recent commands
- Ctrl+c - interrupt a hung process
- COMMON COMMANDS
Command Type: [ Windows ][ Mac/GNU Linux ]
- Create file: [ echo > file ] [ touch file ]
- Create folder: [ mkdir folder ] [ mkdir folder ]
- Change folder: [ cd folder ] [ cd folder ]
- Copy from to: [ copy from to ] [ cp from to ]
- Delete thing: [ del thing ] [ rm thing ]
- Move from to: [ move from to ] [ mv from to ]
- Rename folder: [ rd folder ] [ rm -r folder ]
- List current contents: [ dir ] [ ls ]
- Change to previous folder: [ cd .. ]
- Interrupt a process: [ Ctrl+c ]
- Windows (cmd or powershell)
- systeminfo (output system information, including available memory)
- wmic OS get FreePhysicalMemory /Value (Free memory total)
- wmic computersystem get TotalPhysicalMemory (memory total)
- wmic OS get TotalVisibleMemorySize /Value (Installed total memory)
- systeminfo | findstr "System Memory" (displays memory and system information )
- Linux (terminal)
- free (Free –m (megabytes) –t (total memory))
- free command with -h , displays output in human readable format
- cat /proc/meminfo (report amount of free/used system memory (both physical and swap), shared memory and buffers used by kernel)
- /proc (pseudo filesystem not exist on a disk - kernel creates in memory, provide system information (originally on processes, hence name)
- vmstat -m (virtual memory statistics like 'proc')
- top (Check memory and CPU use per process)
- htop (like 'top' command, memory usage with other details. Header shows CPU usage and RAM /swap usage figures)
- sudo dmidecode -t (hardware information)
DATA CENTRE
- Securing Data Centre
- Day-to-day operation non-dependent on human administrators
- HVAC: Pipes covered in foil insulation. Hot aisle system
- FIRE: inert gas in fire/double-knock - 2 seperate signals activate
- UPS 1 hour running
- Standby generator
- Nightly backups of servers onto disk (for 1 week)
- Weekly backups of netapp to tape (for 4 weeks)
- Relentless Intrusion Detection: alerts if theres an attack(person/virus/worm etc.).
- Firewalls: Software/Hardware/DMZ
- Prevent physical attacks:
- Anti-tailgating/Anti-pass-back turnstile gate. (Single person entry after authentication).
- Single entry point
- Three-factor authentication (ID, fingerprint, voice, retinal scan)
- Padlocks on servers
- CCTV/On-site security
- Hardware
3-Par (accepts a variety of disk):
- SSD (Solid State Drive) fastest, no moving parts so ultra-reliable, most expensive
- SAS (Serial Advanced SCSI) faster, more reliable, more expensive, use more power, lower capacity
- SATA (Serial Advanced Technology Attachment) cheaper, use less power, higher capacity, slower, less reliable (though still very reliable) optimised for capacity
- Sofware
- Virtual Server (aka guest machine): is a software emulation of a physical server.
- In place of 600 servers 4 blade centres managed by hypervisors
- Provisioned (processors, memory, disk space are allocated on demand)
- Can over provision
- Clone servers -taking snap shots (delete when not needed)
- More energy efficient
- RAID (Redundant Array of Independent Disks)
Parity Storage: Simple XOR, rows: odd count = 1, even count= 0:
- SW based: calculations by OS,
- HW based: Independant of OS, Drives know they are mirrored to calculate parity- High performance (especially RAID5 failure)
- Data Duplication