Image Processing for Mechatronics Engineering For senior undergraduate students Academic Year 2017/2018, Winter Semester

Lecture 2: Elementary Image Operations 16.09.2017

Dr. Mohammed Abdel-Megeed Salem Media Engineering Technology, German University in Cairo

Course Info - Contents 1. 2. 3.

Introduction Elementary Image Information and Operations Fundamentals of Signal and Image Processing 1. 2. 3.

4. 5. 6.

Definition, Basic Signals Sampling and Quantization

Image Acqusition and Digitization Sensing and Perception (HVS) and the Color Image Processing Image Operations 1. 2. 3.

Point Image Operations Local Image Operations and Filters Global Image Operation and Transforms

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

2

Report 1 • Based on your own understanding of what is meant by “image”, “image Processing”, “image analysis”, “Computer Vision” and “Computer Graphics”, draw a new Venn diagram similar to the diagram of Castleman. You may include new terms in your diagram, or you may exclude some of the above mentioned terms for simplicity. • Image Processing or Computer Vision have many applications in the field of mechatronics, such as vision-based quality control, robot vision, or autonomous vehicles. Discuss one application to show the details and importance of the role of image processing to achieve the applications’ goals. Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

3

Course Info - Contents 1. 2. 3.

Introduction Elementary Image Information and Operations Fundamentals of Signal and Image Processing 1. 2. 3.

4. 5. 6.

Definition, Basic Signals Sampling and Quantization

Image Acqusition and Digitization Sensing and Perception (HVS) and the Color Image Processing Image Operations 1. 2. 3.

Point Image Operations Local Image Operations and Filters Global Image Operation and Transforms

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

4

Objective • The objective of this lecture is to gain a touchable feeling about the basic information of images and basic operation may be operated on images. This understanding will help us in the coming lecture to study the fundamentals of Signal and Image Processing. • For you to achieve this objective you should try to implement these operations on images created by your own or, when this is not available, on standard set of images. Ex: http://imagedatabase.cs.washington.edu/groundtruth/ Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

5

Simple Example iRGB = imread('fabric.png'); [Rows, Cols, Clrs] = size(iRGB); for (r = 1:Rows) for (c = 1:Cols) iGray(r,c) = (iRGB(r,c,1) + iRGB(r,c,2) + iRGB(r,c,3) )/3; end end

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

6

Report 2-1 • Use the last image RGB to gray example to convert a gray level image to binary image. Use one of your own images. Write down your observations/ comments. • What about converting a gray level image into colored image? Is it possible to restore the original RGB image?

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

7

Image Statistics • An empirical quantities is a measurable real characteristics of a sample (difference to the characteristics of a probability function) – Fg: empirical cumulative distribution function – Pi: frequency distribution

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

8

Image Statistics

m1 = 127,

m1 = 127,

σ2 = 0

σ2 = 127 x 127

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

σ2 = ? m1 = ?

Lecture 2

9

Image Statistics • • • •

Image Size: total number of pixels. Minimum gray level, Maximum gray level, Range, the difference between the maximum and the minimum. • Entropy: a measure of randomness – used to characterize the texture of the image.

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

10

Image Statistics Image Size: 768x1024 = 786432 Minimum gray level: 0 Maxmum gray level: 255 Range: 255 Mean gray level: 142,9 Mode gray level: 160 Median gray level: 151 Entropy: 7.7554

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

12

Report • 2.2 Compute the mean and standard deviation of the given image: • Are the discussed statistical features enough to identify certain image from a collection of images? Support your answer with examples. Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

13

Image Histogram • Histogram is a graph that shows frequency of anything. Histograms usually have bars that represent frequency of occurring of data. • Histogram has two axis: x and y. The x axis contains the event whose frequency we count. The y axis contains frequency.

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

14

Image Histogram • Provides information on grey scale distribution

• Represents the Discrete Probability Function for the colour distribution

ℎ 𝑖 =

𝐼(𝑥, 𝑦) 𝑥

𝑦

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

15

Image Histogram • Most used: normalized histogram with relative frequencies hg of grey values

• For M pixel with G grey values (g = 0, 1,… G-1) and ag as number of pixels with grey value g holds:

and

and

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

16

Image Histogram Interpretation of typical histograms • Homogeneous image: one bar of length 1

• Two level image: two bars • Dark image with low contrast: there are more values of hg for lower g

• Bright image with low contrast: there are more values of hg for larger g • High contrast image: all hg equal (e.g. = 1 / 256)

• Typical foreground / background image: bimodal histogram

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

17

Image Histogram

Marsa Matrouh - Dark

Marsa Matrouh

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Marsa Matrouh - Light

Lecture 2

18

Image Histogram

Marsa Matrouh – High Contrast

Marsa Matrouh

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Marsa Matrouh – Low Contrast

Lecture 2

19

Report 2.3 • Apply the operations of increasing and decreasing the brightness and contrast on one of your own images. Show the histogram of the output images in comparison to the input image and comment the results.

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

20

Image Histogram Nearly bimodal histogram

Meffert and Salem

Image Processing for Mechatronics Engineering

Lecture 21

21

Image Histogram Properties of histograms

• Spatial information is lost • Every image has a special histogram • BUT: every histogram is not assigned to a single image • Example: if objects in an image were rearranged then the histogram is not changed • The structure of an image can not derived from the histogram

• Histogram tells the user if an image is scaled well

Meffert and Salem

Image Processing for Mechatronics Engineering

Lecture 21

22

Image Histogram Two different images and their (identical) histogram

10

Meffert and Salem

Image Processing for Mechatronics Engineering

Lecture 21

23

Image Histogram N-dimensional histogram • Useful for a lot of joint processes

• In image processing (e.g.): multi-channel images • Definition with

number of corresponding pixel pairs

• Example: number of pixels having the value gμ in the matrix for red and gν in the matrix for blue Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

24

Image Statistics - Histogram red

Two-dimensional histogram

number of pixels with the two defined grey values blue Meffert and Salem

Image Processing for Mechatronics Engineering

Lecture 21

25

Image Statistics - Histogram

The baboons colour image BUNT

Meffert and Salem

Image Processing for Mechatronics Engineering

Lecture 21

26

Image Statistics - Histogram

G

B

G

B

R

Color

BUNT

R

Meffert and Salem

Image Processing for Mechatronics Engineering

Lecture 21

27

Image Statistics - Histogram

BUNT

G

R

red

green

5

5 hRlabe l

hGlabe l

512

512

0

100

200 label

300

0

100

200

300

label H1

Meffert and Salem

Image Processing for Mechatronics Engineering

Lecture 21

28

Image Statistics - Histogram

BUNT

R

B

blue

red

5

5

hRlabe l

hBlabe l

512

512

0

100

200 label

Meffert and Salem

300

0

100

200

300

label

Image Processing for Mechatronics Engineering H4

Lecture 21

29

Image Statistics - Histogram

BUNT

G

B

blue

green

5

5

hGlabe l

hBlabe l

512

512

0

100

200 label

Meffert and Salem

300

0

100

200

300

label

Image Processing for Mechatronics Engineering H4

Lecture 21

30

Wikipedia: Mandrillus is the genus of the mandrill and its close relative the drill. These two species are closely related to the baboons, and until recently were lumped together as a single species of baboon. Both Mandrillus species have long furrows on either side of their elongated snouts. The adult male mandrill's furrows are blue, while the furrows of the drill are black. Both species are terrestrial, living on the ground of the rainforest and occasionally grasslands of Central Africa.

Foto: Haremsmännchen

Meffert and Salem

Image Processing for Mechatronics Engineering

Lecture 21

31

Report 2.4 • Find and discuss an application of the 2D Histogram for land use classification in satellite images.

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

32

Image Statistics - Histogram • Cumulative Histogram

The cumulative histogram H(i) at gray level i of an image I is given as: 𝐻 𝑖 =

ℎ(𝑖) 𝑖

This encodes the fraction of pixels with an intensity that is equal to or less than a specific value. Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

33

Image Statistics - Histogram • Cumulative Histogram

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

34

Histogram Equalization k

nj

j 0

n

sk  T (rk )  

k

  pr ( r j ) j 0

• Histogram equalization (HE) results are similar to contrast stretching but offer the advantage of full automation, since HE automatically determines a transformation function to produce a new image with a uniform histogram.

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

36

Basic Image Processing Operations • Image-Processing operations may be divided into 3 classes based on information required to perform the transformation. • 1) Point Operations • process according to the pixel’s value alone (single pixel). • 2) Neighborhood processing • process the pixel in a small neighborhood of pixels around the given pixel. • 3)Transforms • process entire image as one large block

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

37

Point Operations • The gray value of each pixel in the output image g(x, y) depends on the gray value of only the corresponding pixel of the input image f (x,y). Every pixel of f(x, y) with the same gray level maps to a single (usually different) gray value in the output image.

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

38

Image Arithmetic • Image arithmetic is the implementation of standard arithmetic operations. – – – – –

Addition Subtraction Multiplication Division Complement

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

39

Image Arithmetic • Let x is the old gray value, y is the new gray value, c is a positive constant. • • • • •

Addition: y = x + c Subtraction: y = x - c Multiplication: y = cx Division: y = x/c Complement: y= 255 - x

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

40

Image Arithmetic • Image arithmetic has many uses in image processing for example, image subtraction. Image Arithmetic Saturation Rules: • The results of integer arithmetic can easily overflow the data type for storage. • Ex: unit8  [0-255] Result Class Truncated Value 300 uint8 255 -45 uint8 0 10.5 uint8 11 Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

41

Report 3.1 • Discuss different techniques to re-range the new image values of the output images after applying the arithmetic operations, back into the range of [0,255].

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

42

Image Arithmetic - Addition • Lighten the image • Some details may be lost

•+ 128 = Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

43

Image Arithmetic - Subtraction • Darken the image • Some details may be lost

•- 128 Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

= Lecture 2

44

Image Arithmetic - Multiplication Lighten the image Some details may be lost but less than addition

•* 2 Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

= Lecture 2

45

Image Arithmetic - Division Darken the image Some details may be lost but less than subtraction

•/ 2 Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

= Lecture 2

46

Image Arithmetic – Addition vs Multiplication

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

47

Image Arithmetic – Subtraction vs Division

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

48

Image Arithmetic – Complement • Create the negative image

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

49

Image Arithmetic • Image arithmetic functions can be used in combination to perform a series of operations.

I = imread('rice.png');

I2 = imread('cameraman.tif'); K = imdivide(imadd(I,I2), 2); % not recommended Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

52

Image Arithmetic • imlincomb performs all the arithmetic operations in the linear combination in double precision and only rounds and saturates the final result. • K = imlincomb(.5,I,.5,I2); % recommended

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

53

Report 3.2 Apply the image arithmetic operations on one of your own images or on MarsaMatrouhGrey image. Show the histograms and comment the results. 3.3 Use the function imlincomb to turn a new or recent (personal) photo into an old photo.

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

54

Image Arithmetic – Denoising/Sharping

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

55

Image Arithmetic – Fault Detection Where is the defect?

Image g(x,y) (no defect)

Image f (x,y) (with defect)

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

56

Report 3.4 • Consider the previous images of circuits without and with defects. Apply manual alignment to align both images to each other and then apply the subtraction to find the defect.

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

57

Image Arithmetic – Moving Object Detection

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

58

Image Arithmetic – Moving Object Detection • Identify the active cells?

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

59

Thresholding • Segmentation can be achieved simply by thresholding the image at a particular intensity level. • The technique that‘s most frequently employed for determining thresholds involves the histogram of the image. • Problems: – ignores spatial context, – selection of the threshold parameter

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

60

Thresholding

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

61

Thresholding

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

62

Thresholding

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

63

Thresholding

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

64

Thresholding

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

65

Double Thresholding • A larger threshold T2 yields “seeds” of the segmentation, which are accepted in all cases. • They grow in all directions where grey values can be found that exceed some smaller threshold T1. • method does take into account spatial context

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

66

Report 3.5 • Program a small image processing code for fault detection for textile or glass industry. For example your program should be able to detect broken/ splitted/ cracked glass.

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

67

Reading • M. A.-M. Salem, “Multiresolution Image Segmentation”, Humboldt-Universität zu Berlin, 2008, [Chapter 2] • Rafael G. Gonzalaz and Richard E. Woods, Digital Image Processing, 3rd Edition, Pearson Edu., 2008. [Chapter 1] • E.R. Davies, Machine Vision: Theory, Algorithms and Practicalities, Third Edition, Morgan Kaufmann Publishers, 2004. [Chapter 1] • Bovik, A.C.: Handbook of Image and Video Processing. Academic Press, May 2000.

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

68

Contacts Image Processing for Mechatronics Engineering, for senior students, Winter Semester 2017 Dr. Mohammed Abdel-Megeed M. Salem Media Engineering Technology, German University in Cairo Office: C7.311 Tel.: +2 011 1727 1050 Email: [email protected]

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017

Lecture 2

69