P#31 Magic Methods

 Magic or Dunder Methods

Methods having two prefix and suffix underscores in the method name. 

Dunder here means “Double Under (Underscores)”

Sample Method:

# declare our test string class
class Str:

    # magic method to initiate object
    def __init__(self, string):
        self.string = string


# Driver Code
if __name__ == '__main__':
    # object creation
    s1 = Str('Hello Python')

    # print object location
    print(s1)

To print __ methods of int

print(dir(int))

# Result

"""
['__abs__', '__add__', '__and__', '__bool__', '__ceil__', '__class__',
 '__delattr__', '__dir__', '__divmod__', '__doc__', '__eq__', '__float__', 
 '__floor__', '__floordiv__', '__format__', '__ge__', '__getattribute__', 
 '__getnewargs__', '__gt__', '__hash__', '__index__', '__init__', '__init_subclass__',
  '__int__', '__invert__', '__le__', '__lshift__', '__lt__', '__mod__', '__mul__', 
  '__ne__', '__neg__', '__new__', '__or__', '__pos__', '__pow__', '__radd__', '__rand__',
  '__rdivmod__', '__reduce__', '__reduce_ex__', '__repr__', '__rfloordiv__', '__rlshift__',
  '__rmod__', '__rmul__', '__ror__', '__round__', '__rpow__', '__rrshift__', '__rshift__', '__rsub__', '__rtruediv__',
  '__rxor__', '__setattr__', '__sizeof__', '__str__', '__sub__', '__subclasshook__', '__truediv__',
  '__trunc__', '__xor__', 'as_integer_ratio', 'bit_length', 'conjugate', 'denominator', 'from_bytes', 'imag', 
  'numerator', 'real', 'to_bytes']
"""

Important Magic Methods

The following tables list important magic methods in Python 3.

Initialization and Construction	Description
__new__(cls, other)	To get called in an object's instantiation.
__init__(self, other)	To get called by the __new__ method.
__del__(self)	Destructor method.

Unary operators and functions	Description
__pos__(self)	To get called for unary positive e.g. +someobject.
__neg__(self)	To get called for unary negative e.g. -someobject.
__abs__(self)	To get called by built-in abs() function.
__invert__(self)	To get called for inversion using the ~ operator.
__round__(self,n)	To get called by built-in round() function.
__floor__(self)	To get called by built-in math.floor() function.
__ceil__(self)	To get called by built-in math.ceil() function.
__trunc__(self)	To get called by built-in math.trunc() function.

Augmented Assignment	Description
__iadd__(self, other)	To get called on addition with assignment e.g. a +=b.
__isub__(self, other)	To get called on subtraction with assignment e.g. a -=b.
__imul__(self, other)	To get called on multiplication with assignment e.g. a *=b.
__ifloordiv__(self, other)	To get called on integer division with assignment e.g. a //=b.
__idiv__(self, other)	To get called on division with assignment e.g. a /=b.
__itruediv__(self, other)	To get called on true division with assignment
__imod__(self, other)	To get called on modulo with assignment e.g. a%=b.
__ipow__(self, other)	To get called on exponentswith assignment e.g. a **=b.
__ilshift__(self, other)	To get called on left bitwise shift with assignment e.g. a<<=b.
__irshift__(self, other)	To get called on right bitwise shift with assignment e.g. a >>=b.
__iand__(self, other)	To get called on bitwise AND with assignment e.g. a&=b.
__ior__(self, other)	To get called on bitwise OR with assignment e.g. a|=b.
__ixor__(self, other)	To get called on bitwise XOR with assignment e.g. a ^=b.

Type Conversion Magic Methods	Description
__int__(self)	To get called by built-int int() method to convert a type to an int.
__float__(self)	To get called by built-int float() method to convert a type to float.
__complex__(self)	To get called by built-int complex() method to convert a type to complex.
__oct__(self)	To get called by built-int oct() method to convert a type to octal.
__hex__(self)	To get called by built-int hex() method to convert a type to hexadecimal.
__index__(self)	To get called on type conversion to an int when the object is used in a slice expression.
__trunc__(self)	To get called from math.trunc() method.

String Magic Methods	Description
__str__(self)	To get called by built-int str() method to return a string representation of a type.
__repr__(self)	To get called by built-int repr() method to return a machine readable representation of a type.
__unicode__(self)	To get called by built-int unicode() method to return an unicode string of a type.
__format__(self, formatstr)	To get called by built-int string.format() method to return a new style of string.
__hash__(self)	To get called by built-int hash() method to return an integer.
__nonzero__(self)	To get called by built-int bool() method to return True or False.
__dir__(self)	To get called by built-int dir() method to return a list of attributes of a class.
__sizeof__(self)	To get called by built-int sys.getsizeof() method to return the size of an object.

Attribute Magic Methods	Description
__getattr__(self, name)	Is called when the accessing attribute of a class that does not exist.
__setattr__(self, name, value)	Is called when assigning a value to the attribute of a class.
__delattr__(self, name)	Is called when deleting an attribute of a class.

Operator Magic Methods	Description
__add__(self, other)	To get called on add operation using + operator
__sub__(self, other)	To get called on subtraction operation using - operator.
__mul__(self, other)	To get called on multiplication operation using * operator.
__floordiv__(self, other)	To get called on floor division operation using // operator.
__truediv__(self, other)	To get called on division operation using / operator.
__mod__(self, other)	To get called on modulo operation using % operator.
__pow__(self, other[, modulo])	To get called on calculating the power using ** operator.
__lt__(self, other)	To get called on comparison using < operator.
__le__(self, other)	To get called on comparison using <= operator.
__eq__(self, other)	To get called on comparison using == operator.
__ne__(self, other)	To get called on comparison using != operator.
__ge__(self, other)	To get called on comparison using >= operator.

P#30 H/W Internals and Profiling

To know System Information through Python

# Know system Information
# pip install wmi

import wmi

computer = wmi.WMI()
our_system = computer.Win32_ComputerSystem()[0]

print(f"Manufacturer: {our_system.Manufacturer}")
print(f"Model: {our_system. Model}")
print(f"Name: {our_system.Name}")
print(f"NumberOfProcessors: {our_system.NumberOfProcessors}")
print(f"SystemType: {our_system.SystemType}")
print(f"SystemFamily: {our_system.SystemFamily}")
""" Output from my system:
Manufacturer: Dell Inc.

Model: Vostro 3888

Name: MECH1

NumberOfProcessors: 1

SystemType: x64-based PC

SystemFamily: Vostro
"""

Simple System info Python program using platform

# importing module
import platform

# dictionary
info = {}

# platform details
platform_details = platform.platform()

# adding it to dictionary
info["platform details"] = platform_details

# system name
system_name = platform.system()

# adding it to dictionary
info["system name"] = system_name

# processor name
processor_name = platform.processor()

# adding it to dictionary
info["processor name"] = processor_name

# architectural detail
architecture_details = platform.architecture()

# adding it to dictionary
info["architectural detail"] = architecture_details

# printing the details
for i, j in info.items():
   print(i, " - ", j)

"""
platform details  -  Windows-10-10.0.19043-SP0
system name  -  Windows
processor name  -  Intel64 Family 6 Model 165 Stepping 3, GenuineIntel
architectural detail  -  ('64bit', 'WindowsPE')

Process finished with exit code 0
"""

Hardware Information using PSUTIL

import psutil

#Physical cores
print(f"Number of physical cores: {psutil.cpu_count(logical=False)}")
#Logical cores
print(f"Number of logical cores: {psutil.cpu_count(logical=True)}")

#Current frequency
print(f"Current CPU frequency: {psutil.cpu_freq().current}")
#Min frequency
print(f"Min CPU frequency: {psutil.cpu_freq().min}")
#Max frequency
print(f"Max CPU frequency: {psutil.cpu_freq().max}")
# cpu util
#System-wide CPU utilization
print(f"Current CPU utilization: {psutil.cpu_percent(interval=1)}")
#System-wide per-CPU utilization
print(f"Current per-CPU utilization: {psutil.cpu_percent(interval=1, percpu=True)}")

#ram details
#Total RAM
print(f"Total RAM installed: {round(psutil.virtual_memory().total/1000000000, 2)} GB")
#Available RAM
print(f"Available RAM: {round(psutil.virtual_memory().available/1000000000, 2)} GB")
#Used RAM
print(f"Used RAM: {round(psutil.virtual_memory().used/1000000000, 2)} GB")
#RAM usage
print(f"RAM usage: {psutil.virtual_memory().percent}%")

Results

"""
Number of physical cores: 4
Number of logical cores: 8
Current CPU frequency: 3600.0
Min CPU frequency: 0.0
Max CPU frequency: 3600.0
Current CPU utilization: 4.2
Current per-CPU utilization: [15.7, 0.0, 25.0, 0.0, 4.5, 0.0, 1.5, 1.6]
Total RAM installed: 8.33 GB
Available RAM: 1.58 GB
Used RAM: 6.75 GB
RAM usage: 81.0%
"""

Disk and N/W Information using Python

# Disk Usage

import psutil
import platform
from datetime import datetime

def get_size(bytes, suffix="B"):
    """
    Scale bytes to its proper format
    e.g:
        1253656 => '1.20MB'
        1253656678 => '1.17GB'
    """
    factor = 1024
    for unit in ["", "K", "M", "G", "T", "P"]:
        if bytes < factor:
            return f"{bytes:.2f}{unit}{suffix}"
        bytes /= factor

# Disk Information
print("="*40, "Disk Information", "="*40)
print("Partitions and Usage:")
# get all disk partitions
partitions = psutil.disk_partitions()
for partition in partitions:
    print(f"=== Device: {partition.device} ===")
    print(f"  Mountpoint: {partition.mountpoint}")
    print(f"  File system type: {partition.fstype}")
    try:
        partition_usage = psutil.disk_usage(partition.mountpoint)
    except PermissionError:
        # this can be catched due to the disk that
        # isn't ready
        continue
    print(f"  Total Size: {get_size(partition_usage.total)}")
    print(f"  Used: {get_size(partition_usage.used)}")
    print(f"  Free: {get_size(partition_usage.free)}")
    print(f"  Percentage: {partition_usage.percent}%")
# get IO statistics since boot
disk_io = psutil.disk_io_counters()
print(f"Total read: {get_size(disk_io.read_bytes)}")
print(f"Total write: {get_size(disk_io.write_bytes)}")

# N/w
# Network information
print("="*40, "Network Information", "="*40)
# get all network interfaces (virtual and physical)
if_addrs = psutil.net_if_addrs()
for interface_name, interface_addresses in if_addrs.items():
    for address in interface_addresses:
        print(f"=== Interface: {interface_name} ===")
        if str(address.family) == 'AddressFamily.AF_INET':
            print(f"  IP Address: {address.address}")
            print(f"  Netmask: {address.netmask}")
            print(f"  Broadcast IP: {address.broadcast}")
        elif str(address.family) == 'AddressFamily.AF_PACKET':
            print(f"  MAC Address: {address.address}")
            print(f"  Netmask: {address.netmask}")
            print(f"  Broadcast MAC: {address.broadcast}")
# get IO statistics since boot
net_io = psutil.net_io_counters()
print(f"Total Bytes Sent: {get_size(net_io.bytes_sent)}")
print(f"Total Bytes Received: {get_size(net_io.bytes_recv)}")

Result:

"""
======================================== Disk Information ========================================
Partitions and Usage:
=== Device: C:\ ===
  Mountpoint: C:\
  File system type: NTFS
  Total Size: 194.70GB
  Used: 111.24GB
  Free: 83.46GB
  Percentage: 57.1%
=== Device: D:\ ===
  Mountpoint: D:\
  File system type: NTFS
  Total Size: 736.20GB
  Used: 4.03GB
  Free: 732.16GB
  Percentage: 0.5%
Total read: 61.63GB
Total write: 46.68GB
======================================== Network Information ========================================
=== Interface: Local Area Connection* 11 ===
=== Interface: Local Area Connection* 11 ===
  IP Address: 169.254.62.94
  Netmask: 255.255.0.0
  Broadcast IP: None
=== Interface: Local Area Connection* 11 ===
=== Interface: Local Area Connection* 12 ===
=== Interface: Local Area Connection* 12 ===
  IP Address: 169.254.72.255
  Netmask: 255.255.0.0
  Broadcast IP: None
=== Interface: Local Area Connection* 12 ===
=== Interface: Ethernet ===
=== Interface: Ethernet ===
  IP Address: 172.20.24.131
  Netmask: 255.255.0.0
  Broadcast IP: None
=== Interface: Ethernet ===
=== Interface: Wi-Fi 2 ===
=== Interface: Wi-Fi 2 ===
  IP Address: 172.20.29.139
  Netmask: 255.255.0.0
  Broadcast IP: None
=== Interface: Wi-Fi 2 ===
=== Interface: Loopback Pseudo-Interface 1 ===
  IP Address: 127.0.0.1
  Netmask: 255.0.0.0
  Broadcast IP: None
=== Interface: Loopback Pseudo-Interface 1 ===
Total Bytes Sent: 366.06MB
Total Bytes Received: 4.38GB
"""

Measuring Running Time in Python  #1

import timeit
time_test = timeit.repeat('[x**0.5 for x in range(1000)]', number=10000)
print(time_test)

Result:

"""
[1.415492, 1.3847563, 1.3762589999999997, 1.3683869, 1.3789484000000005]
"""

# importing cProfile
import cProfile

cProfile.run("10 + 10")

"""
   3 function calls in 0.000 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.000    0.000 <string>:1(<module>)
        1    0.000    0.000    0.000    0.000 {built-in method builtins.exec}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
"""

Function Profiling code and the corresponding output

# Function Profiling
import cProfile

def f():
   print("Hello Python World Profiling")
cProfile.run('f()')

         5 function calls in 0.000 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.000    0.000 <string>:1(<module>)
        1    0.000    0.000    0.000    0.000 ex1-scit.py:252(f)
        1    0.000    0.000    0.000    0.000 {built-in method builtins.exec}
        1    0.000    0.000    0.000    0.000 {built-in method builtins.print}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

LAB WORK

 Lab Work:

1.  Python Environment setup and Essentials. 

2.  Mathematical computing with Python (NumPy).

3.  Scientific Computing with Python (SciPy).  Numerical & Scientific Python Notes

4.  Data Manipulation with Pandas.

5.  Prediction using Scikit-Learn

6.  Data Visualization in python using matplotlib

Inspiring Industry Visualization Projects

 VISUALISATION

What you imagine is what you see on board/screen.

Information is beautiful. Knowledge is beautiful.

=  information & function & visual form &  story (IFVS)

Refer 1 here 

Refer 2 here

Refer for Data Visualization Good Practices(5)

Best Visualization Samples

Best Dashboard

Anatomy of a Figure

Machine Learning with Matplot

Play with Pandas [Data manipulation with Data]

Case Studies 

MDS

Marine data Science 

LDS - Logistics Data Science

Scipy Intro Tutorial with answers

 Tutorial on scipy intro..

# Tutorial on Scipy intro
# SciPy is a scientific computation library above NumPy.
# SciPy stands for Scientific Python.
# Ref w3schools.com


from scipy import constants
print(constants.liter) # 0.001
print(constants.pi) # 3.141592653589793


import scipy
print(scipy.__version__) # 1.8.1


from scipy import constants
print(constants.yotta)    #1e+24
print(constants.zetta)    #1e+21
print(constants.exa)      #1e+18
print(constants.peta)     #1000000000000000.0
print(constants.tera)     #1000000000000.0
print(constants.giga)     #1000000000.0
print(constants.mega)     #1000000.0
print(constants.kilo)     #1000.0
print(constants.hecto)    #100.0
print(constants.deka)     #10.0
print(constants.deci)     #0.1
print(constants.centi)    #0.01
print(constants.milli)    #0.001
print(constants.micro)    #1e-06
print(constants.nano)     #1e-09
print(constants.pico)     #1e-12
print(constants.femto)    #1e-15
print(constants.atto)     #1e-18
print(constants.zepto)    #1e-21
#
# # Root of an equation ..optimizer
from scipy.optimize import root
from math import cos

def eqn(x):
  return x + cos(x)
myroot = root(eqn, 0)
print(myroot.x)         #[-0.73908513]

#Sparse Matrix to avoid unused elements
import numpy as np
from scipy.sparse import csr_matrix
arr = np.array([0, 0, 0, 0, 0, 1, 1, 0, 2])
print(csr_matrix(arr))
'''
  (0, 5)   1
  (0, 6)   1
  (0, 8)   2
'''

import numpy as np
from scipy.sparse import csr_matrix

arr = np.array([[0, 0, 0], [0, 0, 1], [1, 0, 2]])

mat = csr_matrix(arr)
mat.eliminate_zeros()

print(mat)
'''
 (0, 5)    1
  (0, 6)   1
  (0, 8)   2
  (1, 2)   1
  (2, 0)   1
  (2, 2)   2
'''


#SCIPY sIGNIFICANt aspects

import numpy as np
from scipy.stats import ttest_ind

x1 = np.random.normal(size=50)
x2 = np.random.normal(size=50)
#T-Test
print( ttest_ind(x1, x2))
 #Ttest_indResult(statistic=-0.4602568443314745, pvalue=0.6463514741682714)

# pvalue printing
print(ttest_ind(x1, x2).pvalue) #0.3861625359272315


import numpy as np
from scipy.stats import kstest
x = np.random.normal(size=50)
print(kstest(x, 'norm'))
#KstestResult(statistic=0.07440265161334647, pvalue=0.9254263997475219)


#Statistical Distribuition

import numpy as np
from scipy.stats import describe
x = np.random.normal(size=50)
print(describe(x))
# DescribeResult(nobs=50, minmax=(-2.252010033738992, 1.694245094326367),
# mean=-0.34129189546033056, variance=0.7245922182755077, skewness=0.4445284516357619,
# kurtosis=0.11748436830854025)

# Normality Tests (Skewness and Kurtosis)
'''
Skewness:
A measure of symmetry in data, For normal distributions it is 0.
If it is negative, it means the data is skewed left.
If it is positive it means the data is skewed right.
'''
import numpy as np
from scipy.stats import skew, kurtosis

x = np.random.normal(size=50)

print(skew(x))            #0.25667677411004536   skewed right
print(skew(x))            #-0.49188030505407804  skewed left

'''
Kurtosis:
A measure of whether the data is heavy or lightly tailed with 
respect  to a normal distribution.
Positive kurtosis : heavily tailed.
Negative kurtosis : lightly tailed.
'''
print(kurtosis(x))        #-0.37504350194077585 lightly tailed
print(kurtosis(x))        #0.010624259343263276  heavily tailed

from scipy.stats import normaltest
# To check normal distribution of data
print(normaltest(x)) #NormaltestResult(statistic=7.187162674258679,
# pvalue=0.027499668274458256)