J3UBA

                ah et Linux?.. See on selline Windows, kus.. :)

### TUTVUME LINUKSIGA

##  LIIDESED

  I.

     arvuti ütleb -- kasutaja kuulab
     kasutaja ütleb -- arvuti kuulab

  II.

     arvuti joonistab pildi -- kasutaja näitab kus
     kasutaja pakub-valib (menüü, komboboks, jne..)
     kasutaja joonistab (näpuga, stylusega või hiirega)
     kasutaja žestikuleerib (ruumilised gestuurid, kinex,
       microbit) arvuti püüab mõista, mida temalt tahetakse
     kasutab muid kehakeele võimalusi -- arvuti ei mõista

  III. chat, messenger, terminal

     kirjutad või trükid keeles, millest saab aru vastaspool:
     inglise, mandariini, sh, awk, c, perl, python, veel 1000..
     vastaspool vastab keeles, mida sa oled ise valinud:
       eesti, inglise, vene..

     Kosmonautidel ja astronautidel seda reeglit tuleb täita
     (kohustuslik), suhlemises arvutiga aga seda nimetatakse
     lokaliseerimiseks. (Kui automaattõlge jõuab aktsepteeritava
     tasemeni, dialoog muutub trialoogiks: mina - tõlk - vastaspool)

  IV.  kui te arvate, et viipekeele teadmine läheb teile kasuks, siis reserveerige selle õppimiseks vajalikku aega.


## KASUTAME VIRTUALBOX JA PUTTY.

# VirtualBox:

    OS:            Linux Debian (64-bit)
    RAM:           2048MB
    HD:            Debian9-64.vhd
    Network:       Bridged
    User/password: virtu / virtu
                   root  / root

# PuTTY või KiTTY (fork)

Juhul, kui virtuaalarvutis puudub graafiline liides, putty võimaldab:

  1. kasutada sellist klaviatuuri paigaldust, mida oleme hetkel aktiveerinud Windows-hostil.

  2. kasutada copy-paste hosti ja virtuaalarvuti vahel.

  3. koos X-serveriga (https://sourceforge.net/projects/vcxsrv/ , Windows) käivitada linuksis (kuhu enda graafika ei ole installeeritud) graafilisi programme (chrome, gimp.., code-blocks, mousepad..) Selleks Putty's tuleb lubada X11-forwarding:

  Category -> SSH -> X11 -> Enable X11 forwarding


# KUST VÕTTA LINUKSIT?

1. Kasuta "Linux in a browser" enda arvutis
   (Javascript virtual machine emulator):

    https://jslinux.org/

    https://bellard.org/jslinux/vm.html?
      url=https://bellard.org/jslinux/buildroot-x86.cfg

    https://stackoverflow.com/questions/6030407
      /how-does-linux-emulator-in-javascript-by-fabrice-bellard-work

2. Leia mõni töötav ssh ühendus Linuksi arvutiga:

    https://shells.red-pill.eu/
    https://sdf.org/
    töö, tuttavad, jne.

3. Leia mõnda virtuaalarvutit koolis või näiteks siin:

    https://www.turnkeylinux.org/

4. Kasuta Microsoft Store ja WSL (windows subsystem for Linux)

  otsi:

    windows 10 fighthub

  ning siis windows

    <build_number> ISO

5. Ehita ise Linuksi virtuaalmasina ISO failist kasutades:

     Hyper-V, VirtualBox, VMWare, Parallels, QEmu, TinyEmu, Bochs,..

6. Osta Raspberry-Pi arvutit (al. 30EUR).

7. Või loe siit:

.. an open-source $250 autonomous robotics kit named JetBot (above). It includes a Jetson Nano along with a robot chassis, battery pack, and motors.. With the $99 devkit you get 472 gigaflops of computing powered by a quad-core ARM A57 processor, 128-core Nvidia Maxwell GPU, and 4GB of LPDDR RAM. The Nano also supports a range of popular AI frameworks, including TensorFlow, PyTorch, Caffe, Keras, and MXNet, so most algorithms will be pretty much plug-and-play. And there’s the usual brace of ports and interfaces, including USB-A and B, gigabit Ethernet, and support for microSD storage.

  https://www.theverge.com/circuitbreaker/2019/3/18/18271329
    /nvidia-jetson-nano-price-details-specs-devkit-gdc


# LOKALISEERIMINE, KLAVIATUURI KONFIGUREERIMINE

  # apt-get install keyboard-configuration console-setup
  # dpkg-reconfigure keyboard-configuration
  # service keyboard-setup restart
  # dpkg-reconfigure locales
  UTF-8
  # nano /etc/default/locale
  ..
  LANGUAGE=et_EE:et

To apply new settings.. reboot the whole OS [2].

# LOKALISEERIMISE AJALOOLISED ETAPID

  197x -- ASCII-7, UTF-8 salvrätikul:

    Google: utf-8 brilliant youtube
    https://www.youtube.com/watch?v=MijmeoH9LT4

  200x -- mandriva eestikeelsed man lehed, estobuntu, jne..

  Täna -- speak english.


## PAIGALDAMINE

# Operatsioonsüsteemi installeerimine

  Linux/BSD 1 ja 2 osa..

# Programmi installeerimine ning konfigureerimine.

  1. Repositooriumist (Debian)

     # apt-get update
     # apt-get upgrade
     # apt-cache search <application-name>
     # apt-get install <application-name>
     # apt-get remove <application-name>
     # apt-get autoremove --purge <application-name>
     # apt-get clean

     Andmebaas: /etc/apt/sources.list, /etc/apt/sources.list.d/*

  2. Tarkvarapaketist (software package, vallaste.ee). Näide:

   # url="https://dl.google.com/linux/direct"; \
     url+="/google-chrome-stable_current_amd64.deb"; \
     wget --no-check-certificate $url

   # dpkg -i ./google-chrome-stable_current_amd64.deb

   # apt-get -f install

  3. make abil

     näiteks

     $ wget -c  --no-check-certificate \
         https://www.rarlab.com/rar/rarlinux-x64-5.7.0.tar.gz

     $ tar xf rarlinux-x64-5.7.0.tar.gz

     $ sudo make install

     Lae alla.
     Paki lahti.
     Jälgi readme juhendit
     ..
     # make install

  4. C - algkoodist

    $ gcc mycode.c -o mycode
    $ sudo mv mycode /usr/local/bin/
    $ mycode

Ahvitööd võib vältida skriptide abil. Kui töö seisneb selles, et käivitada ühe käsu teise järel, tuleb panna neid ridu faili. Näiteks eelnev näide failina näeks välja nii:

Faili sisu on nihutatud kaks positsiooni paremale (tundemärk kaks tühikut rea alguses: ^[ ][ ])

Näide 0.

<runme>       see on faili nimi

  #!/bin/sh

  gcc mycode.c -o mycode
  sudo mv mycode /usr/local/bin/
  mycode

EOF           mis tähendab "end of file"

Näide 1. Luua skript, mis näitab kausta sisu, kustutab faili ning uuesti näitab, mis failid on kaustas.

  $ md ~/forest
  $ touch fox
  $ touch rabbit
  $ ed eraser
  a
  #!/bin/sh
  ls
  rm -f $1
  echo it's done..
  ls
  .
  w
  q
  $ chmod +x eraser
  $ ./eraser rabbit

Näide: (suur) Programming Sockets.

  $ cd ~                nüüd oled enda kodukaustas
  $ mkdir sockets       loo kausta nimega "sockets"
  $ cd sockets          mine sinna
  $ rm sockets.txt      kustuta faili sockets.txt
                          isegi kui seda seal pole olemas
  $ ls                  näita, mis failid on kaustas (tühi)
  $ touch sockets.txt   loo tühi fail nimega sockets.txt
  $ ed sockets.txt      lisa sinna clipboardi sisu.
  a
  ..   kleebi (paste) siia ## LISA C. SOCKETS .. faili lõpp
  .    vajuta "Enter" ning trüki uue rea alguses: . (punkt).
       vajuta "Enter".
  w
  q
  $ touch rr.txt        loo tühi fail nimega rr.sh
  $ ed rr.txt           lisa sinna clipboardi sisu.
  a
  ..   kleebi (paste) siia ## LISA B. rr.sh .. EOF
  .    vajuta "Enter" ning trüki uue rea alguses: . (punkt).
       vajuta "Enter".
  w
  q
  $ chmod +x rr.sh       rr.sh on käivitatav fail, ehk programm (skript)
  $ ./rr.sh sockets.txt  käivita skripti
  $ ls                   näita, millised failid on kaustas


## SUDO

  # ls -l /etc/sudoers
  -r--r----- 1 root root 700 Dec  3 20:00 /etc/sudoers

Note: vi -- use w!

  $ su -
  Password:
  # apt-get install sudo
  # cat /etc/sudoers | grep 'root\|virtu'
  ..
  root    ALL=(ALL:ALL) ALL
  virtu   ALL=(ALL) NOPASSWD:ALL


## LINGID

  [1]. https://developer.ibm.com/tutorials/l-lpic1-107-3/
  [2]. https://wiki.debian.org/Keyboard
  [3]. https://www.computerhope.com/unix.htm
  [4]. https://github.com/levskaya/jslinux-deobfuscated
  [5]. Hyper-V series

        https://www.altaro.com/hyper-v/
          hyper-v-for-beginners/

        https://www.altaro.com/hyper-v/
          how-to-create-import-virtual-machines/

        https://www.altaro.com/hyper-v/
          simple-guide-hyper-v-networking/

        https://www.altaro.com/hyper-v/
          storage-and-hyper-v-part-6-how-to-connect/

        https://www.altaro.com/hyper-v-backup/?LP=AltBlog-textmain

  [6]. TinyEMU series (Fabrice Bellard)

        https://bellard.org/tinyemu/buildroot.html

        https://bellard.org/jslinux/tech.html

        https://bellard.org/jslinux/faq.html *

     ------
     *) How can I export a file from the virtual machine ?
        Use the export_file filename shell command.


## LISA A. ESIMESED KÄSUD MIDA TASUB TEADA

00. echo ==================================

  echo

      Print given arguments.

      Print a text message. Note: quotes are optional:

  echo "Hello World"

      Print a message with environment variables:

  echo "My path is $PATH"

      Print a message without the trailing newline:

  echo -n "Hello World"

      Enable interpretation of backslash escapes (special characters):

  echo -e "Column 1\tColumn 2"

01. pwd ===================================

  pwd

  Print name of current/working directory.

  - Print the current directory:
    pwd

  - Print the current directory, and resolve all symlinks (i.e. show the "physical" path):
    pwd -P


02. cd ====================================

  cd

  Change the current working directory.

  - Go to the given directory:
    cd path/to/directory

  - Go to home directory of current user:
    cd

  - Go up to the parent of the current directory:
    cd ..

  - Go to the previously chosen directory:
    cd -


03. mkdir ====================================

  mkdir

  Creates a directory.

  - Create a directory in current directory or given path:
    mkdir directory

  - Create directories recursively (useful for creating nested dirs):
    mkdir -p path/to/directory


04. ls ====================================

  ls

  List directory contents.

  - List files one per line:
    ls -1

  - List all files, including hidden files:
    ls -a

  - Long format list (permissions, ownership, size and modification date) of all files:
    ls -la

  - Long format list with size displayed using human readable units (KB, MB, GB):
    ls -lh

  - Long format list sorted by size (descending):
    ls -lS

  - Long format list of all files, sorted by modification date (oldest first):
    ls -ltr


05. ln ====================================

  ln

  Creates links to files and directories.

  - Create a symbolic link to a file or directory:
    ln -s path/to/file_or_directory path/to/symlink

  - Overwrite an existing symbolic to point to a different file:
    ln -sf path/to/new_file path/to/symlink

  - Create a hard link to a file:
    ln path/to/file path/to/hardlink


06. cp ====================================

  cp

  Copy files and directories.

  - Copy a file to another location:
    cp path/to/file.ext path/to/copy.ext

  - Copy a file into another directory, keeping the filename:
    cp path/to/file.ext path/to/target_parent_directory

  - Copy a directory recursively to another location:
    cp -r path/to/directory path/to/copy

  - Copy a directory recursively, in verbose mode (shows files as they are copied):
    cp -vr path/to/directory path/to/copy

  - Copy the contents of a directory into another directory:
    cp -r path/to/source_directory/* path/to/target_directory

  - Copy text files to another location, in interactive mode (prompts user before overwriting):
    cp -i *.txt path/to/target_directory


07. mv ====================================

  mv

  Move or rename files and directories.

  - Move files in arbitrary locations:
    mv source target

  - Do not prompt for confirmation before overwriting existing files:
    mv -f source target

  - Prompt for confirmation before overwriting existing files, regardless of file permissions:
    mv -i source target

  - Do not overwrite existing files at the target:
    mv -n source target

  - Move files in verbose mode, showing files after they are moved:
    mv -v source target


08. rm ====================================

  rm

  Remove files or directories.

  - Remove files from arbitrary locations:
    rm path/to/file path/to/another/file

  - Recursively remove a directory and all its subdirectories:
    rm -r path/to/directory

  - Forcibly remove a directory, without prompting for confirmation or showing error messages:
    rm -rf path/to/directory

  - Interactively remove multiple files, with a prompt before every removal:
    rm -i file(s)

  - Remove files in verbose mode, printing a message for each removed file:
    rm -v path/to/directory/*


09. touch ====================================

  touch

  Change a file access and modification times (atime, mtime).

  - Create a new empty file(s) or change the times for existing file(s) to current time:
    touch filename

  - Set the times on a file to a specific date and time:
    touch -t YYYYMMDDHHMM.SS filename

  - Use the times from a file to set the times on a second file:
    touch -r filename filename2


10. cmp ====================================

  cmp

  Compare two files.

  - Find the byte number and line number of the first difference between the files:
    cmp file1 file2

  - Find the byte number and differing bytes of every difference:
    cmp -l file1 file2


11. diff ====================================

  diff

  Compare files and directories.

  - Compare files:
    diff file1 file2

  - Compare files, ignoring white spaces:
    diff -w file1 file2

  - Compare files, showing the differences side by side:
    diff -y file1 file2

  - Compare files, showing the differences in unified format (as used by git diff):
    diff -u file1 file2

  - Compare directories recursively:
    diff -r directory1 directory2

  - Compare directories, only showing the names of files that differ:
    diff -rq directory1 directory2


12. patch ====================================

  patch

  Patch a file (or files) with a diff file.
  Note that diff files contain both the target filenames and list of changes.

  - Apply a patch:
    patch < patch_file.diff

  - Apply a patch to current directory:
    patch -p1 < patch_file.diff

  - Apply the reverse of a patch:
    patch -R < patch_file.diff


13. find ====================================

  find

  Find files or directories under the given directory tree, recursively.

  - Find files by extension:
    find root_path -name '*.ext'

  - Find files by matching multiple patterns:
    find root_path -name '*pattern_1*' -or -name '*pattern_2*'

  - Find directories matching a given name:
    find root_path -type d -name *lib*

  - Find files matching path pattern:
    find root_path -path '**/lib/**/*.ext'

  - Run a command for each file, use {} within the command to access the filename:
    find root_path -name '*.ext' -exec wc -l {} \;

  - Find files modified in the last 24-hour period:
    find root_path -mtime -1

  - Find files using case insensitive name matching, of a certain size:
    find root_path -size +500k -size -10M -iname '*.TaR.gZ'

  - Delete files by name, older than 180 days:
    find root_path -name '*.ext' -mtime +180 -delete

  - Find files matching a given pattern, while excluding specific paths:
    find root_path -name '*.py' -not -path '*/site-packages/*'


14. chown ====================================

  chown

  Change user and group ownership of files and directories.

  - Change the owner user of a file/directory:
    chown user path/to/file_or_directory

  - Change the owner user and group of a file/directory:
    chown user:group path/to/file_or_directory

  - Recursively change the owner of a directory and its contents:
    chown -R user path/to/directory

  - Change the owner of a symbolic link:
    chown -h user path/to/symlink

  - Change the owner of a file/directory to match a reference file:
    chown --reference=path/to/reference_file path/to/file_or_directory


15. chgrp ====================================

  chgrp

  Change group ownership of files and directories.

  - Change the owner group of a file/directory:
    chgrp group path/to/file_or_directory

  - Recursively change the owner group of a directory and its contents:
    chgrp -R group path/to/directory

  - Change the owner group of a symbolic link:
    chgrp -h group path/to/symlink

  - Change the owner group of a file/directory to match a reference file:
    chgrp --reference=path/to/reference_file path/to/file_or_directory


16. chmod ====================================

  chmod

  Change the access permissions of a file or directory.

  - Give the [u]ser who owns a file the right to e[x]ecute it:
    chmod u+x file

  - Give the user rights to [r]ead and [w]rite to a file/directory:
    chmod u+rw file_or_directory

  - Remove executable rights from the [g]roup:
    chmod g-x file

  - Give [a]ll users rights to read and execute:
    chmod a+rx file

  - Give [o]thers (not in the file owner's group) the same rights as the group:
    chmod o=g file

  - Change permissions recursively giving [g]roup and [o]thers the abililty to [w]rite:
    chmod -R g+w,o+w directory


17. tar ====================================

  tar

  Archiving utility.
  Often combined with a compression method, such as gzip or bzip.

  - Create an archive from files:
    tar -cf target.tar file1 file2 file3

  - Create a gzipped archive:
    tar -czf target.tar.gz file1 file2 file3

  - Extract an archive in a target directory:
    tar -xf source.tar -C directory

  - Extract a gzipped archive in the current directory:
    tar -xzf source.tar.gz

  - Extract a bzipped archive in the current directory:
    tar -xjf source.tar.bz2

  - Create a compressed archive, using archive suffix to determine the compression program:
    tar -caf target.tar.xz file1 file2 file3

  - List the contents of a tar file:
    tar -tvf source.tar

  - Extract files matching a pattern:
    tar -xf source.tar --wildcards "*.html"


18. bzip2 ====================================

  bzip2

  A block-sorting file compressor.

  - Compress file:
    bzip2 path/to/file_to_compress

  - Decompress file:
    bzip2 -d path/to/compressed_file.bz2

  - Decompress to console:
    bzip2 -dc path/to/compressed_file.bz2


19. gzip ====================================

  gzip

  Compress/uncompress files with gzip compression (LZ77).

  - Compress a file, replacing it with a gzipped compressed version:
    gzip file.ext

  - Decompress a file, replacing it with the original uncompressed version:
    gzip -d file.ext.gz

  - Compress a file specifying the output filename:
    gzip -c file.ext > compressed_file.ext.gz

  - Decompress a gzipped file specifying the output filename:
    gzip -c -d file.ext.gz > uncompressed_file.ext

  - Specify the compression level. 1=Fastest (Worst), 9=Slowest (Best), Default level is 6:
    gzip -9 -c file.ext > compressed_file.ext.gz


20. fdisk ====================================

  fdisk

  A program for managing partition tables and partitions on a hard disk.

  - List partitions:
    fdisk -l

  - Start the partition manipulator:
    fdisk /dev/sda


21. mkfs =====================================

  mkfs.<type> device

  -V  version. Option -V will display version information only when it is the only parameter, otherwise it will work as --verbose.)

  mkfs.ext4 /dev/sda1


22. mkswap ===================================

  mkswap

  Sets up a Linux swap area on a device or in a file.

  - Setup a given partition as swap area:
    sudo mkswap /dev/sdb7

  - Use a given file as swap area:
    sudo mkswap path/to/file

  - Check a partition for bad blocks before creating the swap area:
    sudo mkswap -c /dev/sdb7

  - Specify a label for the file (to allow swapon to use the label):
    sudo mkswap -L swap1 path/to/file

See also: swapon


23. fsck ====================================

  fsck

  Check the integrity of a filesystem or repair it. The filesystem should be unmounted at the time the command is run.

  - Check filesystem /dev/sda, reporting any damaged blocks:
    fsck /dev/sda

  - Check filesystem /dev/sda, reporting any damaged blocks and interactively letting the user choose to repair each one:
    fsck -r /dev/sda

  - Check filesystem /dev/sda, reporting any damaged blocks and automatically repairing them:
    fsck -a /dev/sda


24. dd ====================================

  dd

  Convert and copy a file.

  - Make a bootable usb drive from an isohybrid file (such like archlinux-xxx.iso) and show the progress:
    dd if=file.iso of=/dev/usb_drive status=progress

  - Clone a drive to another drive with 4MB block, ignore error and show progress:
    dd if=/dev/source_drive of=/dev/dest_drive bs=4M conv=noerror status=progress

  - Generate a file of 100 random bytes by using kernel random driver:
    dd if=/dev/urandom of=random_file bs=100 count=1

  - Benchmark the write performance of a disk:
    dd if=/dev/zero of=file_1GB bs=1024 count=1000000

  - Check progress of an ongoing dd operation (Run this command from another shell):
    kill -USR1 $(pgrep ^dd)


25. df ====================================

  df

  Gives an overview of the file system disk space usage.

  - Display all file systems and their disk usage:
    df

  - Display all file systems and their disk usage in human readable form:
    df -h

  - Display the file system and its disk usage containing the given file or directory:
    df path/to/file_or_directory


26. du ====================================

  du

  Disk usage: estimate and summarize file and directory space usage.

  - List the sizes of a directory and any subdirectories, in the given unit (B/KB/MB):
    du -b|k|m path/to/directory

  - List the sizes of a directory and any subdirectories, in human-readable form (i.e. auto-selecting the appropriate unit for each size):
    du -h path/to/directory

  - Show the size of a single directory, in human readable units:
    du -sh path/to/directory

  - List the human-readable sizes of a directory and of all the files and directories within it:
    du -ah path/to/directory

  - List the human-readable sizes of a directory and any subdirectories, up to N levels deep:
    du -h --max-depth=N path/to/directory

  - List the human-readable size of all .jpg files in subdirectories of the current directory, and show a cumulative total at the end:
    du -ch */*.jpg


27. mount ====================================

  mount

  Provides access to an entire filesystem in one directory.

  - Show all mounted filesystems:
    mount

  - Mount a device to a directory:
    mount -t filesystem_type path/to/device_file path/to/target_directory

  - Mount a CD-ROM device (with the filetype ISO9660) to /cdrom (readonly):
    mount -t iso9660 -o ro /dev/cdrom /cdrom

  - Mount all the filesystem defined in /etc/fstab:
    mount -a

  - Mount a specific filesystem described in /etc/fstab (e.g. "/dev/sda1 /my_drive ext2 defaults 0 2"):
    mount /my_drive


28. umount ====================================

  umount

  Unlink a filesystem from its mount point, making it no longer accessible.
  A filesystem cannot be unmounted when it is busy.

  - Unmount a filesystem, by passing the path to the source it is mounted from:
    umount path/to/device_file

  - Unmount a filesystem, by passing the path to the target where it is mounted:
    umount path/to/mounted_directory

  - Unmount all mounted filesystems (except the proc filesystem):
    umount -a


29. ed ====================================

  ed

  The original Unix text editor.

  - Start ed, editing an empty document (which can be saved as a new file in the current directory):
    ed

  - Start ed, editing an empty document, with : as a command prompt indicator:
    ed -p :

  - Start ed editing an existing file (this shows the byte count of the loaded file):
    ed -p : path/to/file

  - Toggle the printing of error explanations. (By default, explanations are not printed and only a ? appears):
    H

  - Add text to the current document. Mark completion by entering a period by itself in a new line:
    a<Enter>text_to_insert<Enter>.

  - Print the entire document (, is a shortcut to the range 1,$ which covers the start to the end of the document):
    ,p

  - Write the current document to a new file (the filename can be omitted if ed was called with an existing file):
    w filename

  - Quit ed:
    q


See also: q


30. nano ====================================

  nano

  Simple, easy to use editor. An enhanced, free Pico clone.

  - Start nano in terminal with {filename}:
    nano filename

  - Enable smooth scrolling:
    nano -S filename

  - Indent new lines to the previous lines' indentation:
    nano -i filename


31. sh ====================================

  sh

  Bourne shell.
  The standard command language interpreter.

  - Start interactive shell:
    sh

  - Execute a command:
    sh -c command

  - Run commands from a file:
    sh file.sh

  - Run commands from STDIN:
    sh -s


32. grep ====================================

  grep

  Matches patterns in input text.
  Supports simple patterns and regular expressions.

  - Search for an exact string:
    grep search_string path/to/file

  - Search in case-insensitive mode:
    grep -i search_string path/to/file

  - Search recursively (ignoring non-text files) in current directory for an exact string:
    grep -RI search_string .

  - Use extended regular expressions (supporting ?, +, {}, () and |):
    grep -E ^regex$ path/to/file

  - Print 3 lines of [C]ontext around, [B]efore, or [A]fter each match:
    grep -C|B|A 3 search_string path/to/file

  - Print file name with the corresponding line number for each match:
    grep -Hn search_string path/to/file

  - Use the standard input instead of a file:
    cat path/to/file | grep search_string

  - Invert match for excluding specific strings:
    grep -v search_string


33. cat ====================================

  cat

  Print and concatenate files.

  - Print the contents of a file to the standard output:
    cat file

  - Concatenate several files into the target file:
    cat file1 file2 > target_file

  - Append several files into the target file:
    cat file1 file2 >> target_file

  - Number all output lines:
    cat -n file


34. wc ====================================

  wc

  Count words, bytes, or lines.

  - Count lines in file:
    wc -l file

  - Count words in file:
    wc -w file

  - Count characters (bytes) in file:
    wc -c file

  - Count characters in file (taking multi-byte character sets into account):
    wc -m file


35. less ====================================

  less

  Open a file for interactive reading, allowing scrolling and search.

  - Open a file:
    less source_file

  - Page down / up:
    <Space> (down), b (up)

  - Go to end / start of file:
    G (end), g (start)

  - Forward search for a string (press n/N to go to next/previous match):
    /something

  - Backward search for a string (press n/N to go to next/previous match):
    ?something

  - Follow the output of the currently opened file:
    F

  - Open the current file in an editor:
    v

  - Exit:
    q


See also: n, q


36. head ====================================

  head

  Output the first part of files.

  - Output the first few lines of a file:
    head -n count_of_lines filename

  - Output the first few bytes of a file:
    head -c size_in_bytes filename

  - Output everything but the last few lines of a file:
    head -n -count_of_lines filename

  - Output everything but the last few bytes of a file:
    head -c -size_in_bytes filename


37. tail ====================================

  tail

  Display the last part of a file.

  - Show last 'num' lines in file:
    tail -n num file

  - Show all file since line 'num':
    tail -n +num file

  - Show last 'num' bytes in file:
    tail -c num file

  - Keep reading file until Ctrl + C:
    tail -f file

  - Keep reading file until Ctrl + C, even if the file is rotated:
    tail -F file


38. tr ====================================

  tr

  Translate characters: run replacements based on single characters and character sets.

  - Replace all occurrences of a character in a file, and print the result:
    tr find_character replace_character < filename

  - Replace all occurrences of a character from another command's output:
    echo text | tr find_character replace_character

  - Map each character of the first set to the corresponding character of the second set:
    tr 'abcd' 'jkmn' < filename

  - Delete all occurrences of the specified set of characters from the input:
    tr -d 'input_characters' < filename

  - Compress a series of identical characters to a single character:
    tr -s 'input_characters' < filename

  - Translate the contents of a file to upper-case:
    tr "[:lower:]" "[:upper:]" < filename

  - Strip out non-printable characters from a file:
    tr -cd "[:print:]" < filename


39. sed ====================================

  sed

  Edit text in a scriptable manner.

  - Replace the first occurrence of a regular expression in each line of a file, and print the result:
    sed 's/regex/replace/' filename

  - Replace all occurrences of an extended regular expression in a file, and print the result:
    sed -r 's/regex/replace/g' filename

  - Replace all occurrences of a string in a file, overwriting the file (i.e. in-place):
    sed -i 's/find/replace/g' filename

  - Replace only on lines matching the line pattern:
    sed '/line_pattern/s/find/replace/' filename

  - Delete lines matching the line pattern:
    sed '/line_pattern/d' filename

  - Print only text between n-th line till the next empty line:
    sed -n 'line_number,/^$/p' filename

  - Apply multiple find-replace expressions to a file:
    sed -e 's/find/replace/' -e 's/find/replace/' filename

  - Replace separator / by any other character not used in the find or replace patterns, e.g., #:
    sed 's#find#replace#' filename


40. awk ====================================

  awk

  A versatile programming language for working on files.

  - Print the fifth column (a.k.a. field) in a space-separated file:
    awk '{print $5}' filename

  - Print the second column of the lines containing "something" in a space-separated file:
    awk '/something/ {print $2}' filename

  - Print the last column of each line in a file, using a comma (instead of space) as a field separator:
    awk -F ',' '{print $NF}' filename

  - Sum the values in the first column of a file and print the total:
    awk '{s+=$1} END {print s}' filename

  - Sum the values in the first column and pretty-print the values and then the total:
    awk '{s+=$1; print $1} END {print "--------"; print s}' filename

  - Print every third line starting from the first line:
    awk 'NR%3==1' filename


41. sort ====================================

  sort

  Sort lines of text files.

  - Sort a file in ascending order:
    sort filename

  - Sort a file in descending order:
    sort -r filename

  - Sort a file in case-insensitive way:
    sort --ignore-case filename

  - Sort a file using numeric rather than alphabetic order:
    sort -n filename

  - Sort the passwd file by the 3rd field, numerically:
    sort -t: -k 3n /etc/passwd

  - Sort a file preserving only unique lines:
    sort -u filename

  - Sort human-readable numbers (in this case the 5th field of ls -lh):
    ls -lh | sort -h -k 5


42. uniq ====================================

  uniq

  Output the unique lines from the given input or file.
  Since it does not detect repeated lines unless they are adjacent, we need to sort them first.

  - Display each line once:
    sort file | uniq

  - Display only unique lines:
    sort file | uniq -u

  - Display only duplicate lines:
    sort file | uniq -d

  - Display number of occurrences of each line along with that line:
    sort file | uniq -c

  - Display number of occurrences of each line, sorted by the most frequent:
    sort file | uniq -c | sort -nr


43. paste ====================================

  paste

  Merge lines of files.

  - Join all the lines into a single line, using TAB as delimiter:
    paste -s file

  - Join all the lines into a single line, using the specified delimiter:
    paste -s -d delimiter file

  - Merge two files side by side, each in its column, using TAB as delimiter:
    paste file1 file2

  - Merge two files side by side, each in its column, using the specified delimiter:
    paste -d delimiter file1 file2

  - Merge two files, with lines added alternatively:
    paste -d '\n' file1 file2


44. passwd ====================================

  passwd

  Passwd is a tool used to change a user's password.

  - Change the password of the current user:
    passwd new_password

  - Change the password of the specified user:
    passwd username new_password

  - Get the current status of the user:
    passwd -S

  - Make the password of the account blank (it will set the named account passwordless):
    passwd -d


45. su ====================================

  su

  Switch shell to another user.

  - Switch to superuser (admin password required):
    su

  - Switch to user {{username}} (password required):
    su username

  - Switch to user {{username}} and simulate a full login shell:
    su - username


46. sudo ====================================

  sudo

  Executes a single command as the superuser or another user.

  - Run a command as the superuser:
    sudo less /var/log/syslog

  - Edit a file as the superuser with your default editor:
    sudo -e /etc/fstab

  - Run a command as another user and/or group:
    sudo -u user -g group id -a

  - Repeat the last command prefixed with "sudo" (only in bash, zsh, etc.):
    sudo !!

  - Launch the default shell with superuser privileges:
    sudo -i


47. ps ====================================

  ps

  Information about running processes.

  - List all running processes:
    ps aux

  - List all running processes including the full command string:
    ps auxww

  - Search for a process that matches a string:
    ps aux | grep string

  - List all processes of the current user in extra full format:
    ps --user $(id -u) -F

  - List all processes of the current user as a tree:
    ps --user $(id -u) f

  - Get the parent pid of a process:
    ps -o ppid= -p pid


48. kill ====================================

  kill

  Sends a signal to a process, usually related to stopping the process.
  All signals except for SIGKILL and SIGSTOP can be intercepted by the process to perform a clean exit.

  - Terminate a program using the default SIGTERM (terminate) signal:
    kill process_id

  - List available signal names (to be used without the SIG prefix):
    kill -l

  - Terminate a program using the SIGHUP (hang up) signal. Many daemons will reload instead of terminating:
    kill -1|HUP process_id

  - Terminate a program using the SIGINT (interrupt) signal. This is typically initiated by the user pressing Ctrl + C:
    kill -2|INT process_id

  - Signal the operating system to immediately terminate a program (which gets no chance to capture the signal):
    kill -9|KILL process_id

  - Signal the operating system to pause a program until a SIGCONT ("continue") signal is received:
    kill -17|STOP process_id

  - Send a SIGUSR1 signal to all processes with the given GID (group id):
    kill -SIGUSR1 -group_id


49. jobs ====================================

  jobs

  BASH builtin for viewing information about processes spawned by the current shell.

  - View jobs spawned by the current shell:
    jobs

  - List jobs and their process ids:
    jobs -l

  - Display information about jobs with changed status:
    jobs -n

  - Display process id of process group leader:
    jobs -p

  - Display running processes:
    jobs -r

  - Display stopped processes:
    jobs -s


50. fg ====================================

  fg

  Run jobs in foreground.

  - Bring most recently suspended background job to foreground:
    fg

  - Bring a specific job to foreground:
    fg job_id


51. bg ====================================

  bg

  Resumes jobs that have been suspended (e.g. using Ctrl + Z), and keeps them running in the background.

  - Resume most recently suspended job and run it in the background:
    bg

  - Resume a specific job (use jobs -l to get its ID) and run it in the background:
    bg job_id


52. dmesg ====================================

  dmesg

  Write the kernel messages to standard output.

  - Show kernel messages:
    dmesg

  - Show kernel error messages:
    dmesg --level err

  - Show kernel messages and keep reading new ones, similar to tail -f (available in kernels 3.5.0 and newer):
    dmesg -w

  - Show how much physical memory is available on this system:
    dmesg | grep -i memory

  - Show kernel messages 1 page at a time:
    dmesg | less

  - Show kernel messages with a timestamp (available in kernels 3.5.0 and newer):
    dmesg -T

  - Show kernel messages in human-readable form (available in kernels 3.5.0 and newer):
    dmesg -H

  - Colorize output (available in kernels 3.5.0 and newer):
    dmesg -L


53. uname ====================================

  uname

  Print details about the current machine and the operating system running on it.
  Note: for additional information about the operating system, try the lsb_release command.

  - Print hardware-related information: machine and processor:
    uname -mp

  - Print software-related information: operating system, release number, and version:
    uname -srv

  - Print the nodename (hostname) of the system:
    uname -n

  - Print all available system information (hardware, software, nodename):
    uname -a


See also: lsb_release


54. ping ====================================

  ping

  Send ICMP ECHO_REQUEST packets to network hosts.

  - Ping host:
    ping host

  - Ping a host only a specific number of times:
    ping -c count host

  - Ping host, specifying the interval in seconds between requests (default is 1 second):
    ping -i seconds host

  - Ping host without trying to lookup symbolic names for addresses:
    ping -n host

  - Ping host and ring the bell when a packet is received (if your terminal supports it):
    ping -a host

  - Also display a message if no response was received:
    ping -O host


55. traceroute ====================================

  traceroute

  Print the route packets trace to network host.

  - Traceroute to a host:
    traceroute host

  - Disable IP address and host name mapping:
    traceroute -n host

  - Specify wait time for response:
    traceroute -w 0.5 host

  - Specify number of queries per hop:
    traceroute -q 5 host

  - Specify size in bytes of probing packet:
    traceroute host 42


56. ip ====================================

  ip

  Show / manipulate routing, devices, policy routing and tunnels.

  - List interfaces with detailed info:
    ip a

  - Display the routing table:
    ip r

  - Show neighbors (ARP table):
    ip n

  - Make an interface up/down:
    ip link set interface up/down

  - Add/Delete an ip address to an interface:
    ip addr add/del ip/mask dev interface

  - Add a default route:
    ip route add default via ip dev interface


57. nslookup ====================================

  nslookup

  Query name server(s) for various domain records.

  - Query your system's default name server for an IP address (A record) of the domain:
    nslookup example.com

  - Query a given name server for a NS record of the domain:
    nslookup -type=NS example.com 8.8.8.8

  - Query for a reverse lookup (PTR record) of an IP address:
    nslookup -type=PTR 54.240.162.118

  - Query for ANY available records using TCP protocol:
    nslookup -vc -type=ANY example.com

  - Query a given name server for the whole zone file (zone transfer) of the domain using TCP protocol:
    nslookup -vc -type=AXFR example.com name_server

  - Query for a mail server (MX record) of the domain, showing details of the transaction:
    nslookup -type=MX -debug example.com

  - Query a given name server on a specific port number for a TXT record of the domain:
    nslookup -port=port_number -type=TXT example.com name_server


58. dig ====================================

  dig

  DNS Lookup utility.

  - Lookup the IP(s) associated with a hostname (A records):
    dig +short hostname.com

  - Lookup the mail server(s) associated with a given domain name (MX record):
    dig +short hostname.com MX

  - Get all types of records for a given domain name:
    dig hostname.com ANY

  - Specify an alternate DNS server to query:
    dig @8.8.8.8 hostname.com

  - Perform a reverse DNS lookup on an IP address (PTR record):
    dig -x 8.8.8.8

  - Find authoritative name servers for the zone and display SOA records:
    dig +nssearch hostname.com

  - Perform iterative queries and display the entire trace path to resolve a domain name:
    dig +trace hostname.com


59. nmap ====================================

  nmap

  Network exploration tool and security / port scanner.
  Some features only activate when Nmap is run with privileges.
  Homepage: https://nmap.org.

  - Try to determine whether the specified hosts are up and what are their names:
    nmap -sn ip_or_hostname optional_another_address

  - Like above, but also run a default 1000-port TCP scan if host seems up:
    nmap ip_or_hostname optional_another_address

  - Also enable scripts, service detection, OS fingerprinting and traceroute:
    nmap -A address_or_addresses

  - Assume good network connection and speed up execution:
    nmap -T4 address_or_addresses

  - Scan a specific list of ports (use -p- for all ports 1-65535):
    nmap -p port1,port2,…,portN address_or_addresses

  - Perform TCP and UDP scanning (use -sU for UDP only, -sZ for SCTP, -sO for IP):
    nmap -sSU address_or_addresses

  - Perform TLS cipher scan against a host to determine supported ciphers and SSL/TLS protocols:
    nmap --script ssl-enum-ciphers address_or_addresses -p 443


60. gcc ====================================

  gcc

  Preprocesses and compiles C and C++ source files, then assembles and links them together.

  - Compile multiple source files into executable:
    gcc source1.c source2.c -o executable

  - Allow warnings, debug symbols in output:
    gcc source.c -Wall -Og -o executable

  - Include libraries from a different path:
    gcc source.c -o executable -Iheader_path -Llibrary_path -llibrary_name

  - Compile source code into Assembler instructions:
    gcc -S source.c

  - Compile source code without linking:
    gcc -c source.c


61. make ====================================

  make

  Task runner for targets described in Makefile.
  Mostly used to control the compilation of an executable from source code.

  - Call the first target specified in the Makefile (usually named "all"):
    make

  - Call a specific target:
    make target

  - Call a specific target, executing 4 jobs at a time in parallel:
    make -j4 target

  - Use a specific Makefile:
    make --file file

  - Execute make from another directory:
    make --directory directory

  - Force making of a target, even if source files are unchanged:
    make --always-make target


62. apt ====================================

  apt

  Package management utility for Debian based distributions.

  - Update the list of available packages and versions (it's recommended to run this before other apt commands):
    sudo apt update

  - Search for a given package:
    apt search package

  - Show information for a package:
    apt show package

  - Install a package, or update it to the latest available version:
    sudo apt install package

  - Remove a package (using purge instead also removes its configuration files):
    sudo apt remove package

  - Upgrade all installed packages to their newest available versions:
    sudo apt upgrade


63. dpkg ====================================

  dpkg

  Debian package manager.

  - Install a package:
    dpkg -i path/to/file.deb

  - Remove a package:
    dpkg -r package_name

  - List installed packages:
    dpkg -l pattern

  - List package contents:
    dpkg -L package_name

  - List contents of a local package file:
    dpkg -c path/to/file.deb

  - Find out which package owns a file:
    dpkg -S file_name


64. crontab = (2**6) ===========================

  crontab

  Schedule cron jobs to run on a time interval for the current user.
  Job definition format: "(min) (hour) (day_of_month) (month) (day_of_week) command_to_execute".

  - Edit the crontab file for the current user:
    crontab -e

  - View a list of existing cron jobs for current user:
    crontab -l

  - Remove all cron jobs for the current user:
    crontab -r

  - Sample job which runs at 10:00 every day. * means any value:
    0 10 * * * path/to/script.sh

  - Sample job which runs every minute on the 3rd of April:
    * * 3 Apr * path/to/script.sh

  - Sample job which runs at 02:30 every Friday:
    30 2 * * Fri path/to/script.sh


## LISA B. rr.sh

file rr.sh

  #!/bin/bash

  # usage: ./rr.sh file2split.txt
  # should have following implants:
  # <filename.extension>
  #   source code
  #   ..
  # EOF

  INSIDE=0;
  FILENAME=dummy

  echo "ATTENTION! EXISTING FILES WOULD BE APPENDED"

  while IFS='' read -r line || [[ -n "$line" ]]; do
      if test "${line:0:1}" = "<"
      then
        if test "${line: -1}" = ">"
        then
          INSIDE=1
          FILENAME=${line:1:-1}
          echo "INSIDE : $INSIDE $FILENAME"
          continue
        fi
      fi

      if test "${line}" = "EOF"
      then
        INSIDE=0
        echo "OUTSIDE: $INSIDE $line"
        continue
      fi

      if test "$INSIDE" -eq 1
      then
        echo "$FILENAME: $line"
        echo "$line" >> $FILENAME
      fi
  done < "$1"

EOF


## LISA C. SOCKETS.TXT (translate.google.com)

Socket programming in Linux
Author: Alexander Shargin
Published: 16/05/2001
Corrected: February 4, 2006
Text version: 1.1

  https://www.rsdn.org/article/unix/sockets.xml

## Introduction

Socket API was first implemented in the Berkley UNIX operating system. Now this software interface is available in almost any version of Unix, including Linux. Although all implementations are somewhat different from each other, the main set of functions in them is the same. Initially, sockets were used in C / C++ programs, but currently many languages ​​provide tools for working with them (Perl, Java, etc.).

Sockets provide a very powerful and flexible interprocess communication (IPC) mechanism. They can be used to organize the interaction of programs on one computer, over a local network or via the Internet, which allows you to create distributed applications of varying complexity. In addition, they can be used to interact with programs running other operating systems. For example, under Windows there is an interface Window Sockets, designed on the basis of the socket API. Below we will see how easy it is to adapt an existing Unix program to work under Windows.

Sockets support many standard network protocols (their specific list depends on the implementation) and provide a unified interface for working with them. Most often sockets are used to work in IP networks. In this case, they can be used to interact with applications not only using specially developed, but also using standard protocols - HTTP, FTP, Telnet, etc. For example, you can write your own web browser or a web server that can serve multiple clients simultaneously .

As you can see, sockets are a very powerful and convenient tool for network programming. In this article I will show how to use them. Starting with the concept of a socket and the most basic functions for working with it, we will gradually move on to the discussion of more complex topics. In particular, we will consider the use of low-level sockets, various ways of organizing parallel customer service, the use of standard Internet protocols and interaction with programs running under the Microsoft Windows operating system.

NOTE

Most of the material presented in the article applies to the entire Unix OS family. Nevertheless, all the following facts and demonstration programs were checked only under Linux, therefore the name of this OS is in the title of the article.

## socket API basics

# Socket concept

A socket is the end point of network communications. It is something like a "portal" through which you can send bytes to the outside world. The application simply writes the data to the socket; their further buffering, sending and transporting is performed by the used protocol stack and network equipment. Reading data from a socket is similar.

In a program, a socket is identified by a descriptor — it is simply a variable of type int. The program receives a descriptor from the operating system when creating a socket, and then passes it to the socket API services to specify a socket on which to perform one or another action.

Socket Attributes

Three attributes are associated with each socket: domain, type, and protocol. These attributes are set when creating a socket and remain unchanged throughout its existence. To create a socket, use the socket function, which has the following prototype.

  #include <sys/types.h>
  #include <sys/socket.h>

  int socket(int domain, int type, int protocol);

The domain defines the address space in which the socket is located, and the set of protocols that are used to transfer data. The most commonly used domains are Unix and Internet, defined by the AF_UNIX and AF_INET constants, respectively (the AF prefix means "address family" - "address family"). Setting AF_UNIX uses Unix I / O for data transfer. In this case, sockets are used for interprocess communication on one computer and are not suitable for networking. The constant AF_INET corresponds to the Internet-domain. Sockets hosted in this domain can be used to work on any IP network. There are other domains (AF_IPX for Novell protocols, AF_INET6 for the new modification of the IP protocol - IPv6, etc.), but in this article we will not consider them.

The type of socket determines how data is transmitted over the network. More commonly used:

  + SOCK_STREAM. Transmission of data stream with pre-connection. A reliable data transmission channel is provided, in which fragments of a sent block are not lost, not reordered, or duplicated. Since this type of sockets is the most common, until the end of the section we will only talk about it. The remaining types will be devoted to separate sections.

  + SOCK_DGRAM. Data transmission in the form of separate messages (datagrams). Pre-installing the connection is not required. Data exchange is faster, but is unreliable: messages can be lost in transit, duplicated and reordered. You can send messages to multiple recipients (multicasting) and broadcast (broadcasting).

  + SOCK_RAW. This type is assigned to low-level (so-called "raw") sockets. Their difference from conventional sockets is that with their help the program can take on the formation of some headers added to the message.

Please note that not all domains allow specifying an arbitrary socket type. For example, only the SOCK_STREAM type is used with the Unix domain. On the other hand, for the Internet domain you can specify any of the listed types. In this case, TCP is used to implement SOCK_STREAM, UDP is used to implement SOCK_DGRAM, and SOCK_RAW is used for low-level work with IP, ICMP, etc.

Finally, the last attribute defines the protocol used to transfer the data. As we have just seen, often the protocol is uniquely determined by the domain and type of the socket. In this case, you can pass 0 as the third parameter to the socket function, which corresponds to the default protocol. However, sometimes (for example, when working with low-level sockets) it is required to specify the protocol explicitly. Numeric protocol IDs depend on the selected domain; they can be found in the documentation.

# Addresses

Before transmitting data through a socket, it must be associated with an address in the selected domain (this procedure is called the socket name). Sometimes the binding is implicit (inside the connect and accept functions), but it must be done in all cases. The type of address depends on your chosen domain.

!!! In a Unix domain, this is a text string — the name of the file through which data is exchanged.

In the Internet domain, the address is specified by a combination of an IP address and a 16-bit port number. The IP address identifies the host on the network, and the port specifies the specific socket on that host. TCP and UDP protocols use different port spaces.

To bind the socket to some address explicitly, use the bind function. Its prototype is:

  #include <sys/types.h>
  #include <sys/socket.h>

  int bind(int sockfd, struct sockaddr *addr, int addrlen);

The first parameter is the socket descriptor that we want to bind to the specified address. The second parameter, addr, contains a pointer to the structure with the address, and the third parameter is the length of this structure. Let's see what it is.

  struct sockaddr {
      unsigned short sa_family; // Address family, AF_xxx
      char sa_data [14]; // 14 bytes to store the address
  };

The sa_family field contains the domain identifier, the same as the first parameter of the socket function. Depending on the value of this field, the contents of the sa_data array are interpreted differently. Of course, working with this array directly is not very convenient, so you can use instead of sockaddr one of the alternative structures of the form sockaddr_XX (XX is the suffix denoting the domain: "un" - Unix, "in" - Internet, etc.). When passing to the bind function, a pointer to this structure is cast to a pointer to sockaddr. For example, consider the sockaddr_in structure.

  struct sockaddr_in {
      short int sin_family;        // Address family
      unsigned short int sin_port; // Port number
      struct in_addr sin_addr;     // IP address
      unsigned char sin_zero[8];   // "Addition" to the size
                                   // sockaddr structures
  };

Here, the sin_family field corresponds to the sa_family field in sockaddr, the port number is recorded in sin_port, and the host's IP address is entered in sin_addr. The sin_addr field itself is a structure that looks like:

  struct in_addr {
      unsigned long s_addr;
  };

Why did it take to enclose only one field in the structure? The fact is that before in_addr was a union (union), containing a much larger number of fields. Now, when there is only one field left in it, it continues to be used for backward compatibility.

And one more important note. There are two ways to store bytes in a word and a double word. One of them is called the host byte order, the other is the network byte order of the byte storage. When specifying the IP address and port number, you need to convert the number from host order to network. To do this, use the functions htons (Host TO Network Short) and htonl (Host TO Network Long). The inverse transform is performed by the functions ntohs and ntohl.

NOTE

On some machines (this does not apply to the PC), the host order and the network byte storage order are the same. However, conversion functions are best used there, as this will improve the portability of the program. This will not affect the performance in any way, since the preprocessor will remove all the "extra" calls of these functions, leaving them only where the transformation is really necessary.

# Connection establishment (server)

Establishing a connection on the server side consists of four stages, none of which can be omitted. First, a socket is created and bound to a local address. If the computer has several network interfaces with different IP addresses, you can only accept connections from one of them by passing its address to the bind function. If you are ready to connect to clients via any interface, set the address to the constant INADDR_ANY. As for the port number, you can specify a specific number or 0 (in this case, the system itself will select an arbitrary unused port number at the moment).

In the next step, a queue of connection requests is created. In this case, the socket is transferred to the mode of waiting for requests from clients. All this is performed by the listen function.

  int listen(int sockfd, int backlog);

The first parameter is the socket descriptor, and the second is the size of the request queue. Every time a regular client tries to connect to the server, its request is queued, as the server may be busy processing other requests. If the queue is full, all subsequent requests will be ignored. When the server is ready to serve the next request, it uses the accept function.

  #include <sys/socket.h>

  int accept(int sockfd, void *addr, int *addrlen);

The accept function creates a new socket for communication with the client and returns its descriptor. The sockfd parameter sets the listening socket. After the call, it remains in the listening state and can accept other connections. The structure referenced by addr records the address of the client socket that established the connection to the server. In the variable addressed by the pointer addrlen, the size of the structure is initially written; The accept function writes there the length that was actually used. If you are not interested in the client’s address, you can simply pass NULL as the second and third parameters.

Notice that the new socket received from accept is associated with the same address as the listening socket. At first it may seem strange. But the fact is that the address of a TCP socket does not have to be unique in the Internet domain. Only connections should be unique, which are identified by the two socket addresses between which data is exchanged.

# Connection establishment (client)

On the client side, the connect function is used to establish the connection, which has the following prototype.

  #include <sys/types.h>
  #include <sys/socket.h>

  int connect(int sockfd, struct sockaddr *serv_addr, int addrlen);

Here, sockfd is the socket that will be used to communicate with the server, serv_addr contains a pointer to the structure with the server address, and addrlen contains the length of this structure. Usually, a socket is not required to be pre-bound to a local address, since the connect function will do it for you, picking up a suitable free port. You can force a port number to a client socket by using bind before calling connect. This should be done when the server connects to only clients using a specific port (examples of such servers are rlogind and rshd). In other cases, it is easier and more reliable to provide the system with a choice of port for you.

# Data exchange

Once the connection is established, you can begin the exchange of data. For this, the send and recv functions are used. In Unix, you can also use the read and write file functions for working with sockets, but they have fewer features and, besides, they will not work on other platforms (for example, under Windows), so I do not recommend using them.

The send function is used to send data and has the following prototype.

  int send(int sockfd, const void *msg, int len, int flags);

Here, sockfd is, as always, the socket descriptor through which we send data, msg is a pointer to the data buffer, len is the length of the buffer in bytes, and flags is a set of bit flags that control the operation of the function (if no flags are used, pass the functions 0). Here are some of them (a full list can be found in the documentation):

  + MSG_OOB. Instructs to send data as urgent (out of band data, OOB). The concept of urgent data allows you to have two parallel data channels in one connection. Sometimes it is convenient. For example, Telnet uses urgent data to send commands like Ctrl + C. Currently, they are not recommended for use due to compatibility issues (there are two different standards for their use, as described in RFC793 and RFC1122). It is safer to simply create a separate connection for urgent data.

  + MSG_DONTROUTE. Disable packet routing. Lower transport layers may ignore this flag.

The send function returns the number of bytes that were actually sent (or -1 in case of an error). This number may be less than the specified buffer size. If you want to send the entire buffer, you have to write your function and call send in it until all the data have been sent. It may look something like this.

  int sendall(int s, char *buf, int len, int flags)
  {
      int total = 0;
      int n;

      while(total < len)
      {
          n = send(s, buf+total, len-total, flags);
          if(n == -1) { break; }
          total += n;
      }

      return (n==-1 ? -1 : total);
  }

Using sendall is no different from using send, but it sends the entire buffer with the entire data.

To read data from the socket, use the recv function.

  int recv(int sockfd, void *buf, int len, int flags);

In general, its use is similar to send. It also accepts a socket handle, a pointer to a buffer, and a set of flags. The MSG_OOB flag is used to receive urgent data, and MSG_PEEK allows you to "peep" data received from a remote host without deleting it from the system buffer (this means that the next time you use recv, you will receive the same data). A complete list of flags can be found in the documentation. Similar to send, recv returns the number of bytes read, which may be less than the buffer size. You can easily write your own recvall function that fills the entire buffer. There is another special case in which recv returns 0. This means that the connection was broken.

# Close socket

When finished exchanging data, close the socket using the close function. This will break the connection.

  #include <unistd.h>

  int close(int fd);

You can also prohibit data transfer in one direction using shutdown.

  int shutdown(int sockfd, int how);

The how parameter can be one of the following values:

    0 - disable reading from socket
    1 - disable writing to socket
    2 - ban both

Although after calling shutdown with a how parameter equal to 2, you can no longer use a socket to exchange data, you still need to call close to release the associated system resources.

# Error processing

Until now, I have not said a word about errors that can occur (and often occur) in the process of working with sockets. So: if something went wrong, all the functions we considered return -1, writing the error code to the global variable errno. Accordingly, you can analyze the value of this variable and take actions to restore the normal operation of the program, without interrupting its execution. Or you can simply issue a diagnostic message (it is convenient to use the perror function for this), and then terminate the program using exit. This is what I will do in demo examples.

# Debugging programs

Beginner programmers often ask how to debug a network program if there is no network at hand. It turns out you can do without it. It is enough to run the client and the server on the same machine, and then use the interface loopback interface address for the connection. In the program, it corresponds to the constant INADDR_LOOPBACK (do not forget to apply the htonl function to it!). Packets sent to this address are not included in the network. Instead, they are passed to the TCP / IP stack of protocols as they were just received. This simulates the presence of a virtual network in which you can debug your network applications.

For simplicity, I will use the inner loop interface in demo examples.

# Echo client and echo server

Now that we have learned the basic functions for working with sockets, it's time to see how they are used in practice. For this, I wrote two small demo programs. The echo client sends the message "Hello there!" and displays the server response. Its code is shown in Listing 1. The echo server reads everything that the client sends to it, and then simply sends the received data back. Its code is shown in Listing 2.

Listing 1. Echo client.

<echoclient.c>

  #include <stdio.h>
  #include <stdlib.h>
  #include <fcntl.h> // for open
  #include <unistd.h> // for close
  #include <sys/types.h>
  #include <sys/socket.h>
  #include <netinet/in.h>

  char message[] = "Hello there!\n";
  char buf[sizeof(message)];

  int main()
  {
      int sock;
      struct sockaddr_in addr;

      sock = socket(AF_INET, SOCK_STREAM, 0);
      if(sock < 0)
      {
          perror("socket");
          exit(1);
      }

      addr.sin_family = AF_INET;
      addr.sin_port = htons(3425); // or any other port ...
      addr.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
      if(connect(sock, (struct sockaddr *)&addr, sizeof(addr)) < 0)
      {
          perror("connect");
          exit(2);
      }

      send(sock, message, sizeof(message), 0);
      recv(sock, buf, sizeof(message), 0);

      printf(buf);
      close(sock);

      return 0;
  }

EOF

Listing 2. Echo server.

<echoserver.c>

  #include <stdio.h>
  #include <stdlib.h>
  #include <fcntl.h> // for open
  #include <unistd.h> // for close
  #include <sys/types.h>
  #include <sys/socket.h>
  #include <netinet/in.h>

  int main()
  {
      int sock, listener;
      struct sockaddr_in addr;
      char buf[1024];
      int bytes_read;

      listener = socket(AF_INET, SOCK_STREAM, 0);
      if(listener < 0)
      {
          perror("socket");
          exit(1);
      }

      addr.sin_family = AF_INET;
      addr.sin_port = htons(3425);
      addr.sin_addr.s_addr = htonl(INADDR_ANY);
      if(bind(listener, (struct sockaddr *)&addr, sizeof(addr)) < 0)
      {
          perror("bind");
          exit(2);
      }

      listen(listener, 1);

      while(1)
      {
          sock = accept(listener, NULL, NULL);
          if(sock < 0)
          {
              perror("accept");
              exit(3);
          }

          while(1)
          {
              bytes_read = recv(sock, buf, 1024, 0);
              if(bytes_read <= 0) break;
              send(sock, buf, bytes_read, 0);
          }

          close(sock);
      }

      return 0;
  }

EOF

# Datagram exchange

As already mentioned, datagrams are rarely used in programs. In most cases, transmission reliability is critical for the application, and instead of inventing their own reliable protocol over UDP, programmers prefer to use TCP. However, sometimes datagrams are useful. For example, they are convenient to use when broadcasting audio or video over the network in real time, especially when broadcasting.

Since datagrams do not need to be connected to exchange data, it is much easier to use them. Having created a socket using socket and bind, you can immediately use it to send or receive data. To do this, you need the sendto and recvfrom functions.

  int sendto(int sockfd, const void *msg, int len, unsigned int flags,
             const struct sockaddr *to (int tolen);

  int recvfrom (int sockfd, void *buf, int len, unsigned int flags,
                struct sockaddr *from, int *fromlen);

The sendto function is very similar to send. The two additional parameters to and tolen are used to specify the recipient address. To set the address, the sockaddr structure is used, as is the case with the connect function. The recvfrom function works like recv. After receiving the next message, it writes its address to the structure referenced by from, and the recorded number of bytes into the variable pointed to by fromlen. As we know, the accept function works in a similar way.

Connected datagram sockets (connected datagram sockets) bring some confusion. The fact is that for a socket of type SOCK_DGRAM, you can also call the connect function, and then use send and recv to exchange data. It should be understood that no connection is established. The operating system simply remembers the address you passed to the connect function, and then uses it when sending data. Note that an attached socket can only receive data from the socket with which it is connected.

To illustrate the datagram exchange process, I wrote two small programs, sender (Listing 3) and receiver (Listing 4). The first sends messages "Hello there!" and "Bye bye!", and the second gets them and prints on the screen. The sender program demonstrates the use of both a regular and an attached socket, and the receiver uses a normal one.

Listing 3. The sender program

<sender.c>

  #include <stdio.h>
  #include <stdlib.h>
  #include <fcntl.h> // for open
  #include <unistd.h> // for close
  #include <sys/types.h>
  #include <sys/socket.h>
  #include <netinet/in.h>

  char msg1[] = "Hello there!\n";
  char msg2[] = "Bye bye!\n";

  int main()
  {
      int sock;
      struct sockaddr_in addr;

      sock = socket(AF_INET, SOCK_DGRAM, 0);
      if(sock < 0)
      {
          perror("socket");
          exit(1);
      }

      addr.sin_family = AF_INET;
      addr.sin_port = htons(3425);
      addr.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
      sendto(sock, msg1, sizeof(msg1), 0,
             (struct sockaddr *)&addr, sizeof(addr));

      connect(sock, (struct sockaddr *)&addr, sizeof(addr));
      send(sock, msg2, sizeof(msg2), 0);

      close(sock);

      return 0;
  }

EOF

Listing 4. The receiver program

<receiver.c>

  #include <stdio.h>
  #include <stdlib.h>
  #include <fcntl.h> // for open
  #include <unistd.h> // for close
  #include <sys/types.h>
  #include <sys/socket.h>
  #include <netinet/in.h>
  #include <stdio.h>

  int main()
  {
      int sock;
      struct sockaddr_in addr;
      char buf[1024];
      int bytes_read;

      sock = socket(AF_INET, SOCK_DGRAM, 0);
      if(sock < 0)
      {
          perror("socket");
          exit(1);
      }

      addr.sin_family = AF_INET;
      addr.sin_port = htons(3425);
      addr.sin_addr.s_addr = htonl(INADDR_ANY);
      if(bind(sock, (struct sockaddr *)&addr, sizeof(addr)) < 0)
      {
          perror("bind");
          exit(2);
      }

      while(1)
      {
          bytes_read = recvfrom(sock, buf, 1024, 0, NULL, NULL);
          buf[bytes_read] = '\0';
          printf(buf);
      }

      return 0;
  }

EOF

# Using low-level sockets

Low-level sockets open up new horizons for you. They provide the programmer with complete control over the contents of the packets that are sent on the network. On the other hand, they are more difficult to use and have poor tolerance. That is why they should be used only when necessary. For example, one cannot do without them when developing system utilities like ping and traceroute.

First of all, find out how low-level sockets differ from normal sockets. Working with ordinary sockets, you transmit "clean" data to the system, and she takes care of adding the necessary headers to them (and sometimes also trailers). For example, when you send a message through a UDP socket, a UDP header is added to it, then an IP header, and at the very end, a hardware protocol header that is used on your local network (for example, Ethernet). The result is a frame shown in Figure 1.

  Ethernet header
  IP Header
  UDP header
  Data

Figure 1

Low-level sockets allow you to buffer the headers of some protocols into the data buffer. For example, you can include a TCP or UDP header in your message, letting the system generate an IP header for you, or you can create all the headers yourself. Of course, in this case you will have to study the work of the relevant protocols and strictly observe the format of their titles, otherwise the program will not work.

When working with low-level sockets, you have to specify the protocol in the third parameter of the socket function, the headers of which you want to access. The constants for the basic Internet protocols are declared in the netinet / in.h file. They look like IPPROTO_XXX, where XXX is the protocol name: IPPROTO_TCP, IPPROTO_UDP, IPPROTO_RAW (in the latter case, you will have the opportunity to work with raw IP and form IP headers manually).

All numeric data in the headers should be recorded in a network format. Therefore, do not forget to use the functions htons and htonl.

To illustrate all this with an example, I rewrote the sender program from the previous section using low-level UDP sockets. At the same time, I had to manually create the UDP header of the message being sent. I chose UDP for the example, because this protocol looks quite simple for this protocol (Figure 2).

  Sender port (16 bit)              Recipient port (16 bit)
  length (header + data) (16 bits)  Checksum (16 bits)

Figure 2

The example code is shown in Listing 5. I want to draw your attention to a few points. First, I did not set the port number in the sockaddr_in structure. Since this number is contained in the UDP header, nothing depends on the sin_port field. Secondly, I wrote zero as a checksum so as not to bore you with its calculation. The UDP protocol is unreliable in nature, so it allows for such liberty. But other protocols (for example, IP) may not allow. Finally, note that all UDP header data is formatted using htons.

Note: Run as root

  $ ./sender-ll
  socket: Operation not permitted
  $ sudo ./sender-ll

Listing 5. A sender program using low-level sockets.

<sender-ll.c>

  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>
  #include <fcntl.h> // for open
  #include <unistd.h> // for close
  #include <sys/types.h>
  #include <sys/socket.h>
  #include <netinet/in.h>

  struct UdpHeader
  {
      u_short src_port;
      u_short targ_port;
      u_short length;
      u_short checksum;
  };

  char message[] = "Hello there!\n";
  char msgbuf[1024];

  int main()
  {
      int sock;
      struct sockaddr_in addr;
      struct UdpHeader header;

      sock = socket(AF_INET, SOCK_RAW, IPPROTO_UDP);
      if(sock < 0)
      {
          perror("socket");
          exit(1);
      }

      addr.sin_family = AF_INET;
      addr.sin_addr.s_addr = htonl(INADDR_LOOPBACK);

      header.targ_port = htons(3425);
      header.length = htons(sizeof(header)+sizeof(message));
      header.checksum = 0;

      memcpy((void *)msgbuf, (void *)&header, sizeof(header));
      memcpy(
          (void *)(msgbuf+sizeof(header)),
          (void *)message, sizeof(message)
      );

      sendto(sock, msgbuf, sizeof(header)+sizeof(message), 0,
             (struct sockaddr *)&addr, sizeof(addr));

      close(sock);

      return 0;
  }

EOF

# Functions for working with addresses and DNS

In this section, we will discuss several functions, without which you can write a case study, but without which a real program is unlikely to cost. Since domain names are widely used to identify hosts on the Internet, we need to examine the mechanism for converting them into IP addresses. In addition, we will explore several handy auxiliary functions.

IP addresses are usually written as four numbers separated by dots. To convert an address written in this format to a number and vice versa, the inet_addr, inet_aton, and inet_ntoa family of functions are used.

  #include <sys/socket.h>
  #include <netinet/in.h>
  #include <arpa/inet.h>

  int inet_aton(const char *cp, struct in_addr *in_p);
  unsigned long int inet_addr(const char *cp);
  char *inet_ntoa(struct in_addr in);

The inet_addr function is often used in programs. It takes a string and returns the address (already with the network byte order). The problem with this function is that the -1 value returned by it in the event of an error is at the same time the correct address is 255.255.255.255 (broadcast address). That is why it is now recommended to use the new inet_aton function (Ascii TO Network). The inet_ntoa (Network TO Ascii) function is used for the inverse transform. Both of these functions work with addresses in a network format. Note that in case of an error, they return 0, not -1.

To convert a domain name to an IP address, use the gethostbyname function.

  #include <netdb.h>

  struct hostent *gethostbyname(const char * name);

This function gets the host name and returns a pointer to the structure with its description. Consider this structure in more detail.

  struct hostent {
      char    *h_name;
      char    **h_aliases;
      int     h_addrtype;
      int     h_length;
      char    **h_addr_list;
  };
  #define h_addr h_addr_list[0]

    h_name.      Hostname
    h_aliases.   An array of strings containing host aliases.
                 Ends with a null value.
    h_addrtype.  Type of address. For Internet-domain - AF_INET.
    h_length.    The length of the address in bytes.
    h_addr_list. An array containing the addresses of all network interfaces
                 host. It ends with zero. Note,
                 that the bytes of each address are stored in network order,
                 therefore, htonl does not need to be called.

As you can see, gethostbyname returns fairly complete information. If we are interested in the address of the host, we can select it from the h_addr_list array. Often the very first address is taken (as we saw above, the special macro h_addr is defined to refer to it). To determine the host name at the address, the function gethostbyaddr is used. Instead of a string, it receives an address (in the form of sockaddr) and returns a pointer to the same hostent structure. Using these two functions, you need to remember that they report an error differently than the others: instead of a pointer, NULL is returned, and the extended error code is written to the global variable h_errno (and not errno). Accordingly, to display a diagnostic message, use herror instead of perror.

It should be noted that the functions gethostbyname and gethostbyaddr return a pointer to a static memory area. This means that each new access to one of these functions will result in overwriting the data obtained during the previous access.

In conclusion, we will consider another family of useful functions — gethostname, getsockname, and getpeername.

  #include <unistd.h>

  int gethostname(char *hostname, size_t size);

The gethostname function is used to get the name of the local host. You can then convert it to an address with gethostbyname. This gives us a way at any time to programmatically get the address of the machine on which our program runs, which can be useful in many cases.

  #include <sys/socket.h>

  int getpeername(int sockfd, struct sockaddr *addr, int *addrlen);

The getpeername function allows you to find the socket address at the "other end" of the connection at any time. It gets a handle to the socket connected to the remote host, and writes the address of that host to the structure pointed to by addr. The actual number of bytes written is placed at addrlen (do not forget to write the size of the addr structure there before calling getpeername). The resulting address, if necessary, can be converted to a string using inet_ntoa or gethostbyaddr. The getsockname function is intended to be the reverse of getpeername and allows you to determine the socket address at "our end" of the connection.

# Parallel customer service

The next important issue that we have to discuss is parallel customer service. This problem becomes relevant when the server needs to handle a large number of requests. Of course, on a machine with one processor, true parallelism cannot be achieved. But even on one machine, you can achieve a significant performance gain. Suppose the server has sent some data to the client and is waiting for confirmation. As long as it travels over the network, the server could easily be busy with other clients. To implement such a service algorithm, there are many ways, but most often two of them are used.

Method 1

This method involves creating a child process to serve each new customer. In this case, the parent process is engaged only in listening to the port and receiving connections. To achieve this behavior, immediately after accept, the server calls the fork function to create a child process (I assume that you are familiar with the fork function; if not, consult the documentation). Next, the value returned by this function is analyzed. In the parent process, it contains the ID of the child, and in the child process it is zero. Using this sign, we move to the next call to accept in the parent process, and the child process serves the client and ends (_exit).

Using this technique, our echo server will be overwritten, as shown in Listing 6.

Listing 6. Echo server (version 2, fork)

<echoserver2.c>

  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>
  #include <fcntl.h>  // for open
  #include <unistd.h> // for close
  #include <sys/types.h>
  #include <sys/socket.h>
  #include <netinet/in.h>

  int main()
  {
      int sock, listener;
      struct sockaddr_in addr;
      char buf[1024];
      int bytes_read;

      listener = socket(AF_INET, SOCK_STREAM, 0);
      if(listener < 0)
      {
          perror("socket");
          exit(1);
      }

      addr.sin_family = AF_INET;
      addr.sin_port = htons(3425);
      addr.sin_addr.s_addr = INADDR_ANY;
      if(bind(listener, (struct sockaddr *)&addr, sizeof(addr)) < 0)
      {
          perror("bind");
          exit(2);
      }

      listen(listener, 1);

      while(1)
      {
          sock = accept(listener, NULL, NULL);
          if(sock < 0)
          {
              perror("accept");
              exit(3);
          }

          switch(fork())
          {
          case -1:
              perror("fork");
              break;

          case 0:
              close(listener);
              while(1)
              {
                  bytes_read = recv(sock, buf, 1024, 0);
                  if(bytes_read <= 0) break;
                  send(sock, buf, bytes_read, 0);
              }

              close(sock);
              _exit(0);

          default:
              close(sock);
          }
      }

      close(listener);

      return 0;
  }

EOF

The obvious advantage of this approach is that it allows you to write very compact, understandable programs in which the connection setup code is separated from the customer service code. Unfortunately, it has its drawbacks. Firstly, if there are a lot of customers, creating a new process to service each of them may be too expensive. Secondly, this method implicitly implies that all clients are serviced independently of each other. However, this may not be the case. If, for example, you are writing a chat server, then your main task is to support the interaction of all the clients that have joined it. In these conditions, the boundaries between processes will become a serious obstacle for you. In this case, you should seriously consider another way of serving customers.

Method 2

The second method is based on the use of non-blocking sockets (nonblocking sockets) and the select function. First, let's look at what non-blocking sockets are. The sockets that we have used so far have been blocking. This name means that your program is blocked for the duration of the operation with such a socket. For example, if you called recv, but there is no data at your end of the connection, then your program will “fall asleep” while waiting for them to come. A similar situation occurs when you call accept, and the connection request queue is empty. This behavior can be changed using the fcntl function.

  #include <unistd.h>
  #include <fcntl.h>
  .
  .
  sockfd = socket(AF_INET, SOCK_STREAM, 0);
  fcntl(sockfd, F_SETFL, O_NONBLOCK);
  .
  .

This simple operation turns a socket into a non-blocking one. Calling any function with such a socket will return control immediately. Moreover, if the requested operation was not completed to the end, the function will return -1 and write the value EWOULDBLOCK to errno. To wait for the operation to complete, we can poll all our sockets in a loop until some function returns a value other than EWOULDBLOCK. Once this happens, we can run the next operation with this socket and return to our polling cycle. Such tactics (called polling in the English literature) are efficient, but very inefficient, since the processor time is wasted on multiple (and unsuccessful) polls.

To remedy the situation, use the select function. This feature allows you to monitor the status of several file descriptors (and in Unix they include sockets) at the same time.

  #include <sys/time.h>
  #include <sys/types.h>
  #include <unistd.h>

  int select(int n, fd_set *readfds, fd_set *writefds,
                  fd_set *exceptfds, struct timeval *timeout);

  FD_CLR(int fd, fd_set *set);
  FD_ISSET(int fd, fd_set *set);
  FD_SET(int fd, fd_set *set);
  FD_ZERO(int fd);

The select function works with three sets of descriptors, each of which is of type fd_set. Sockets descriptors are written to the readfds set, from which we need to read data (listening sockets are added to the same set). A lot of writefds should contain the descriptors of the sockets to which we are going to write, and exceptfds the descriptors of the sockets that need to be monitored for an error. If you are not interested in any set, you can pass NULL instead of a pointer to it. As for the other parameters, in n you need to record the maximum value of the descriptor for all sets plus one, and in timeout - the timeout value. The timeval structure has the following format.

  struct timeval {
      int tv_sec;  // seconds
      int tv_usec; // microseconds
  };

The "microsecond" field looks impressive. But in practice, you will not achieve this accuracy of time measurement using select. Real accuracy will be in the region of 100 milliseconds.

Now let's do a lot of descriptors. The FD_XXX functions shown above are provided for working with them; their use completely hides from us the details of the internal device fd_set. Consider their purpose.

- clears set

    FD_ZERO (fd_set *set)

- adds the fd handle to the set

    FD_SET (int fd, fd_set *set)

- removes fd descriptor from set

    FD_CLR (int fd, fd_set *set)

- checks if the fd descriptor is contained in the set

    FD_ISSET (int fd, fd_set *set)

If at least one socket is ready to perform the specified operation, select returns a non-zero value, and all descriptors that led to the function “triggering” are written into the corresponding sets. This allows us to analyze the descriptors contained in the sets and perform the necessary actions on them. If the timeout has occurred, select returns zero, and in the case of an error, -1. Extended code is written to errno.

Programs that use non-blocking sockets with select are very confusing. If in the case of fork, we build the program logic, as if the client is only one, here the program is forced to track the descriptors of all clients and work with them in parallel. To illustrate this technique, I once again rewrote the echo server using select. The new version is shown in Listing 7. Note that this program, unlike all the others, is written in C ++ (and not C). I used the set class from the STL library of the C ++ language to make it easier to work with a set of descriptors and make it more understandable.

Listing 7. Echo server (version 3, non-blocking sockets and select).

<echoserver3.cpp>

  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>
  #include <fcntl.h> // for open
  #include <unistd.h> // for close
  #include <sys/types.h>
  #include <sys/socket.h>
  #include <sys/time.h>
  #include <netinet/in.h>
  #include <stdio.h>
  #include <unistd.h>
  #include <fcntl.h>
  #include <algorithm>
  #include <set>
  using namespace std;

  int main()
  {
      int listener;
      struct sockaddr_in addr;
      char buf[1024];
      int bytes_read;

      listener = socket(AF_INET, SOCK_STREAM, 0);
      if(listener < 0)
      {
          perror("socket");
          exit(1);
      }

      fcntl(listener, F_SETFL, O_NONBLOCK);

      addr.sin_family = AF_INET;
      addr.sin_port = htons(3425);
      addr.sin_addr.s_addr = INADDR_ANY;
      if(bind(listener, (struct sockaddr *)&addr, sizeof(addr)) < 0)
      {
          perror("bind");
          exit(2);
      }

      listen(listener, 2);

      set<int> clients;
      clients.clear();

      while(1)
      {
          // Fill a set of sockets
          fd_set readset;
          FD_ZERO(&readset);
          FD_SET(listener, &readset);

          for(
              set<int>::iterator it = clients.begin();
              it != clients.end();
              it++
          )
              FD_SET(*it, &readset);

          // Set the timeout
          timeval timeout;
          timeout.tv_sec = 15;
          timeout.tv_usec = 0;

          // Waiting for event in one of the sockets
          int mx = max(
              listener,
              *max_element(clients.begin(), clients.end())
          );
          if(select(mx+1, &readset, NULL, NULL, &timeout) <= 0)
          {
              perror("select");
              exit(3);
          }

          // Determine the type of event and
          // perform the appropriate actions.
          if(FD_ISSET(listener, &readset))
          {
              // Received a new request to connect, use accept
              int sock = accept(listener, NULL, NULL);
              if(sock < 0)
              {
                  perror("accept");
                  exit(3);
              }

              fcntl(sock, F_SETFL, O_NONBLOCK);

              clients.insert(sock);
          }

          for(
              set<int>::iterator it = clients.begin();
              it != clients.end();
              it++
          )
          {
              if(FD_ISSET(*it, &readset))
              {
                  // Received data from the client, read them
                  bytes_read = recv(*it, buf, 1024, 0);

                  if(bytes_read <= 0)
                  {
                      // Connection broken,
                      // remove the socket from the set
                      close(*it);
                      clients.erase(*it);
                      continue;
                  }

                  // Send the data back to the client
                  send(*it, buf, bytes_read, 0);
              }
          }
      }

      return 0;
  }

EOF

# Work on standard protocols

As I said earlier, sockets can be used when writing applications that use the Internet application layer protocols (HTTP, FTP, SMTP, etc.). In this case, the interaction between the client and the server occurs in the same way as the interaction between the echo client and the echo server in our example. The difference is that the data exchanged between the client and the server are interpreted in accordance with the prescriptions of the relevant protocol.

For example, a web server can work according to the following algorithm.

  Create a listening socket and bind it to port 80 (the standard port for the HTTP server).

  + We accept the next request for connection.

  + Read the HTTP request from the client (it has a standard format and is described in RFC2616).

  + We process the request and send a response to the client, which also has a standard format.

  + Break the connection.

A web browser that is a client with respect to the web server can use a similar algorithm.

  + Connect to the server at the specified address.

  + We send him an HTTP request.

  + We receive and process the server's response (for example, we format and display the resulting HTML page).

  + Break the connection.

As we see, there is nothing complicated or fundamentally new in the work on standard protocols.

# Breakthrough off the platform

In the world of the Internet, the interaction of programs running on different platforms occurs quite often. So, almost every second Internet Explorer connects to the Apache web server, and the next Netscape Navigator connects to IIS quite calmly. That is why it is very useful to write programs so that they can be easily transferred to other platforms. In this section, we will look at how to port Linux-based programs using sockets to the Windows platform.

The list of major differences between socket API and Winsock API looks like this.

  + In Windows, the set of header files is significantly reduced. As a matter of fact, you need to include only one file winsock.h (or winsock2.h, if you want to use the advanced features of Winsock 2).

  + In Windows, the Winsock library needs to be explicitly initialized before accessing any other functions from it. This is done using the WSAStartup function. In addition, there is a WSACleanup function that should be called when the work with sockets is completed.

  + As we know, in Linux, socket descriptors are of type int. In Windows, sockets are not file descriptors, so their own SOCKET type is entered for them. Although this type is declared as u_int, it should not be relied upon in the program.

  + Windows does not use file input / output functions (read and write) for working with sockets. Closesocket is used instead of close.

  + In Windows, the global variable errno is not used. Instead, the last error code is stored by the system for each thread separately. To get it, use the WSAGetLastError function.

  + Windows introduced additional constants that should be used instead of specific numbers. So, the values ​​returned by the Winsock functions should be compared with the constants INVALID_SOCKET or SOCKET_ERROR, and not with -1.

If we rewrite our echo client with the given features of the Winsock API, and then compile it under Windows (for example, using Visual C ++), it will be able to interact with the echo server running under Linux. Thus, sockets can solve the problem of cross-platform interaction between the two applications.

Unfortunately, the socket API and Winsock differences are not limited to the above list. When porting more complex, "advanced" programs, more fundamental problems start to arise. For example, under Windows, there are limitations in supporting low-level sockets (they first appeared in the Winsock 2 specification, and the ability to directly manipulate IP headers is available only under Windows 2000). In addition, problems may arise with functions that are not directly related to the socket API. So, in Windows there is no direct analog of the fork function, and in order to organize parallel customer service, you will have to resort to other means.

## Conclusion

In this article, we looked at a number of important aspects of socket programming. Thus, we have laid a solid foundation for further research in this area. Of course, a large number of details remained beyond the scope of our conversation. But now you can independently gather the missing information from the Linux man pages and from your own practical experience. Good luck.


$ cat _bld

echoclient.c
echoserver2.c
echoserver3.cpp
echoserver.c
receiver.c
sender.c
sender-ll.c


$ cat _bld | sed 's/^/gcc /' | sed 's/ \([a-z0-9]\+\)\(\.cp*\)$/ \1\2 -o \1/'


## SELLE AASTA APRILLI NALJAD:

Ühetähelised akronüümid:

  Aastaaeg   Nädalapäev
  GER        EST           JAP

  Frühling   Esmaspäev      1 tsuki
  Sommer     Teisipäev      2 gatsu
  Herbst     Kolmapäev      3 tsuki
  Winter     Neljapäev      4 tsuki
             Reede          5 tsuki
             Laupäeval      6 tsuki
             Pühapäev       7 tsuki
                            8 tsuki
                            9 tsuki
                           10 tsuki
                           11 tsuki
                           12 tsuki

------

CV -- cirriculum vitae
CW -- crossword

------

И вот что любопытно, уже находясь в точке невозврата,
он, (свято веря в тайну "самосознания" которым якобы
была наделена его "прошивка"), с упорством продолжал
поиск своих  бывших  неудач  в  неизученной  области
субквантовых  слоев  параллельной  вселенной,  тогда
как для весьма точного совпадения с  "экспериментом"
вполне хватало и закона Ома, пары законов Кирхгофа..
да принципа математической индукции.

Отсюда вывод: Жизнь в одномерном мире требует
развитого чувства доминанты.

------

Informaatikas pole sellist mõistet nagu
puu (tree), on ainult põõsad (bushes)

          R          R -- root
         / \         N -- node (nodes)
        L   N        L -- leaf (leaves)
           / \
          L   L

------

Privaatsuse illusioon:

Klient: Mõtled, et turvariski pole, kuid on (turvaauk).
Häkker: Õieti mõtled:  RiSKi polegi, on AUK.
        Kes see idioot võiks olla, kes ütleks välja,
          kus nimelt on see auk?
        See ongi meil kõige rangem reegel:
        käitu kliendiga nii, et ta ei teaks august midagi.
        Kui kõik sellest reeglist kinni peavad,
        siis tal polegi mingit turva..riski :)

------

Kohtume tulnukaga teiselt planeedilt:

  1. kasutame teda ära nii,
     nagu meie oleks osavad manipulaatorid.
  2. ravime ta terveks nagu kogenud arstid.
  3. nüüd tutvume kellega meil üldse tegemist on (oli) :)

------

Teie parool on "parool", kuid vangla-koodis.
     (torukoputamisleviedastus*, näiteks "OK":
      34-25: I-I-I--I-I-I-I---I-I--I-I-I-I-I-I :)

*)   0-1-2-3-4-5->x  kus:  "I"  Kop, KnocK, туК
     1 a b c d e           "-"  vaikus, millisekundites
     2 f g h i k
     3 l m n o p
     4 q r s t u
     5 v w x y z
     |
   y v

