DS Syllabus for AMET

Course Code	:	UEAI002
Course Title	:	Introduction to Data Science
Number of Credits	:	3 (L: 2; T: 0; P: 2)
Course Category	:	DAS

 

Course Objective:

·         To Provide the knowledge and expertise to become a proficient data scientist;

·         Demonstrate and understanding of statistics and machine learning concepts that        

        are vital for data science;

·         Produce Python code to statistically analyse a dataset;

·         Critically evaluate data visualisations based on their design and use 

        For  communicating stories from data;

 

Course Contents:

Module 1: [ 7 Lectures]

 

Introduction to Data Science,

 

 

Different Sectors using Data science

https://mail.google.com/mail/u/0/?tab=rm&ogbl#inbox/FMfcgzGpFWQsDnsGDsLLmPxJHTLkDfsz,

 

Purpose and Components of Python in Data Science.

 https://towardsdatascience.com/top-10-reasons-why-you-need-to-learn-python-as-a-data-scientist-e3d26539ec00

 

 

 

 

 

Module 2: [ 7 Lectures]

Data Analytics Process, Knowledge Check, Exploratory Data Analysis (EDA), EDA- Quantitative technique, EDA- Graphical Technique, Data Analytics Conclusion and Predictions.

Module 3: [ 11 Lectures]

Feature Generation and Feature Selection (Extracting Meaning from Data)- Motivating application: user (customer) retention- Feature Generation (brainstorming, role of domain expertise, and place for imagination)- Feature Selection algorithms.

Module 4: [ 10 Lectures]

Data Visualization- Basic principles, ideas and tools for data visualization, Examples of inspiring (industry) projects- Exercise: create your own visualization of a complex dataset.

Module 5: [ 7 Lectures]

Applications of Data Science, Data Science and Ethical Issues- Discussions on privacy, security, ethics- A look back at Data Science- Next-generation data scientists.

Lab Work:

1.  Python Environment setup and Essentials.

2.  Mathematical computing with Python (NumPy).

3.  Scientific Computing with Python (SciPy).

4.  Data Manipulation with Pandas.

5.  Prediction using Scikit-Learn

6.  Data Visualization in python using matplotlib

Text Books/References:

 

1.      Business Analytics: The Science of Data - Driven Decision Making, U Dinesh Kumar, John Wiley & Sons.

 

2.      Introducing Data Science: Big Data, Machine Learning, and More, Using Python Tools, Davy Cielen, John Wiley & Sons.

 

3.      Joel Grus, Data Science from Scratch, Shroff Publisher/O’Reilly Publisher Media

 

4.      Annalyn Ng, Kenneth Soo, Numsense! Data Science for the Layman, Shroff Publisher Publisher

 

5.      Cathy O’Neil and Rachel Schutt. Doing Data Science, Straight Talk from The Frontline. O’Reilly Publisher.

 

6.      Jure Leskovek, Anand Rajaraman and Jeffrey Ullman. Mining of Massive Datasets. v2.1, Cambridge University Press.

 

7.      Jake VanderPlas, Python Data Science Handbook, Shroff Publisher/O’Reilly Publisher Media.

 

8.      Philipp Janert, Data Analysis with Open Source Tools, Shroff Publisher/O’Reilly Publisher Media.

 

Tutorials

https://realpython.com/tutorials/data-science/

https://www.w3schools.com/datascience/

https://learn.theprogrammingfoundation.org/getting_started/intro_data_science/

 

 

 

P#01.1 Fundas Opeartors

 # Python divides the operators in the following groups:
# Arithmetic operators   + -  * / % ** // 
# Assignment operators  =   +=  -= *= /= %= //= **=  &= |= ^= >>= <<= 
# Comparison operators == != > < >= <=
# Logical operators and or not 
# Identity operators is is not 
# Membership operators in not in 
# Bitwise operators & |  ^ ~ << >>

# Arithmetic operators   + -  * / % ** // 

x,y = 911,247

print('x=',x)

print('y=',y)

print()

print('x+y=',x+y)

print('x-y=',x-y)

print('x*y=',x*y)

print('x/y=',x/y)

print()

print('3**2=',3**2) #Exponentiation

print()

print('x%y=',x%y) #Modulus

print('x//y=',x//y)# FloorDivision

z = (y * ( x // y) + (x % y))

print(z)        # check Modulus + floor division

Output:

x= 911
y= 247
x+y= 1158
x-y= 664
x*y= 225017
x/y= 3.688259109311741
3**2= 9
x%y= 170
x//y= 3
911

# Assignment operators = += -= *= /= %= //= **= &= |= ^= >>= <<=
a = 5
b = 3
print(a,b)
print()
a += b
print('a += b ' ,a)
a -= b
print('a -= b ' ,a)
a *= b
print('a -= 5 ' ,a)
a /= b
print('a /= b ' ,a)
print()
a **= b
print('a **= b ' ,a)
a %= b
print('a %= b ' ,a)
a //= b
print('a //= b ' ,a)
print()
a=5
b=3
a &= b
print('a &= b ' ,a)
a |= b
print('a |= b ' ,a)
a ^= b
print('a ^= b ' ,a)
print()
a=5
b=3
a <<=b
print('a <<= b ' ,a)
a >>=b
print('a >>= b ' ,a)
output
5 3
a += b 8
a -= b 5
a -= 5 15
a /= b 5.0
a **= b 125.0
a %= b 2.0
a //= b 0.0
a &= b 1
a |= b 3
a ^= b 0
a <<= b 40
a >>= b 5
--------------------------------
# Comparison Operators
a = 15
b = 13
print(a,b)
print()
print('a == b ' ,a==b)
print('a != b ' ,a!=b)
print('a > b ' ,a>b)
print('a < b ' ,a<b)
print('a >= b ' ,a>=b)
print('a <= b ' ,a<=b)
print()
output:
15 13
a == b False
a != b True
a > b True
a < b False
a >= b True
a <= b False
-----------------------------
#Logical operators and or not
a=6
print ('a>4 and a <11 ',a>4 and a<11)
print()
print('a>4 or a <11 ', a>4 or a<11)
print()
print('not(a>4 and a <11) ', not(a>4 and a<11))
print()
output:
a>4 and a <11 True
a>4 or a <11 True
not(a>4 and a <11) False

P#26 natural Language processing

 NLTK

Very easy to tokenize the statement with NLTK of python

import nltk

word_data ='Sitting pretty,impatient, work from home, 

we can work from home'

tokens = nltk.word_tokenize(word_data)

print(tokens)

#['Sitting', 'pretty', ',', 'impatient', ',', 'work', 

'from', 'home', ',', 'we', 'can', 'work', 'from', 'home']

Another Exercise:

sentence_data = "Johhny Jonny Yes papa. 

Eating Sugar No. NO. papa "

nltk_tokens = nltk.sent_tokenize(sentence_data)

print(nltk_tokens) # ['Johhny Jonny Yes papa.', 

'Eating Sugar No.', 'NO.', 'papa']

Happy Learning @AMET ODL!!!

P#25 DATE TIME

 Python - Date and Time

import datetime

print('The Date Today is  :', datetime.datetime.today())


date_today = datetime.date.today()

print(date_today)

print('This Year   :', date_today.year)

print('This Month    :', date_today.month)

print('Month Name:',date_today.strftime('%B'))

print('This Week Day    :', date_today.day)

print('Week Day Name:',date_today.strftime('%A'))

"""
The Date Today is  : 2022-04-15 22:56:14.461386
2022-04-15
This Year   : 2022
This Month    : 4
Month Name: April
This Week Day    : 15
Week Day Name: Friday
"""

DATE TIME ARITHMETIC

import datetime

# First Date
day1 = datetime.date(2020, 2, 12)
print('day1:', day1.ctime())

#  Second Date
day2 = datetime.date(2019, 8, 18)
print ('day2:', day2.ctime())

# Difference between the dates
print('Number of Days:', day1 - day2)

date_today = datetime.date.today()

# Create a delta of Four Days
no_of_days = datetime.timedelta(days=4)

# Use Delta for Past Date -
before_four_days = date_today - no_of_days
print('Before Four Days:', before_four_days)

# Use Delta for future Date +
after_four_days = date_today + no_of_days
print('After Four Days:', after_four_days)

"""
day1: Wed Feb 12 00:00:00 2020
day2: Sun Aug 18 00:00:00 2019
Number of Days: 178 days, 0:00:00
Before Four Days: 2022-04-11
After Four Days: 2022-04-19
"""

Happy Learning at AMET ODL!👦👧👥👭👬

P#25 Pandas Data Frame set_index() and reset_index()

 Data Frame set_index() and reset_index()

These methods are very useful. We can dynamically change re index and reset index with out any problem.

import pandas as  pd


drinks= pd.read_csv('http://bit.ly/drinksbycountry')

print(drinks.shape) #(193, 6)

print(drinks.index) #RangeIndex(start=0, stop=193, step=1)

print(drinks.columns)
"""Index(['country', 'beer_servings', 'spirit_servings', 'wine_servings',
       'total_litres_of_pure_alcohol', 'continent'],
      dtype='object')
"""

drinks.set_index('continent',inplace=True)


print(drinks.index)

"""
Index(['Asia', 'Europe', 'Africa', 'Europe', 'Africa', 'North America',
       'South America', 'Europe', 'Oceania', 'Europe',
       ...
       'Africa', 'North America', 'South America', 'Asia', 'Oceania',
       'South America', 'Asia', 'Asia', 'Africa', 'Africa'],
      dtype='object', name='continent', length=193)
"""

print(drinks.head())

"""
              country  ...  total_litres_of_pure_alcohol
continent               ...                              
Asia       Afghanistan  ...                           0.0
Europe         Albania  ...                           4.9
Africa         Algeria  ...                           0.7
Europe         Andorra  ...                          12.4
Africa          Angola  ...                           5.9

[5 rows x 5 columns]
"""

print(drinks.shape) #(193, 5)

print(drinks.loc['Europe', 'spirit_servings'])
"""
continent
Europe    132
Europe    138
....
Europe    237
Europe    126
Name: spirit_servings, dtype: int64
"""

drinks.index.name = None


print(drinks.index)

"""
Index(['Asia', 'Europe', 'Africa', 'Europe', 'Africa', 'North America',
       'South America', 'Europe', 'Oceania', 'Europe',
       ...
       'Africa', 'North America', 'South America', 'Asia', 'Oceania',
       'South America', 'Asia', 'Asia', 'Africa', 'Africa'],
      dtype='object', length=193)

"""

drinks.index.name='continent'

drinks.reset_index(inplace=True)  # Resetting Index inplace

print(drinks.head(2))

"""
 continent      country  ...  wine_servings  total_litres_of_pure_alcohol
0      Asia  Afghanistan  ...              0                           0.0
1    Europe      Albania  ...             54                           4.9

[2 rows x 6 columns]
"""

The above code snippets and output shows , how it is very easy to operate in Data Frames

Happy Learning @ AMET ODL!!!!

P#24 Pandas loc() and iloc()

Differences between loc() and iloc()

With these methods, we can simulate SQL for filtering rows or columns. Let us dive in to this exercise. 

import pandas as pd

df = pd.read_csv('uforeports.csv')

# df =pd.read_csv('http://bit.ly/uforeports') Alternateively directly from web

print(df.shape)  # to print rows xcolumns  (18241,  5)

print(df.describe)
"""
<bound method NDFrame.describe of                        City Colors Reported  ... State              Time
0                    Ithaca             NaN  ...    NY    6/1/1930 22:00
1               Willingboro             NaN  ...    NJ   6/30/1930 20:00
2                   Holyoke             NaN  ...    CO   2/15/1931 14:00
3                   Abilene             NaN  ...    KS    6/1/1931 13:00
4      New York Worlds Fair             NaN  ...    NY   4/18/1933 19:00
...                     ...             ...  ...   ...               ...
18236            Grant Park             NaN  ...    IL  12/31/2000 23:00
18237           Spirit Lake             NaN  ...    IA  12/31/2000 23:00
18238           Eagle River             NaN  ...    WI  12/31/2000 23:45
18239           Eagle River             RED  ...    WI  12/31/2000 23:45
18240                  Ybor             NaN  ...    FL  12/31/2000 23:59

[18241 rows x 5 columns]>
"""



# print(df.head(5))  # to Print top 5 rows
"""
                   City Colors Reported Shape Reported State             Time
0                Ithaca             NaN       TRIANGLE    NY   6/1/1930 22:00
1           Willingboro             NaN          OTHER    NJ  6/30/1930 20:00
2               Holyoke             NaN           OVAL    CO  2/15/1931 14:00
3               Abilene             NaN           DISK    KS   6/1/1931 13:00
4  New York Worlds Fair             NaN          LIGHT    NY  4/18/1933 19:00
"""

# print(df.tail(5))  # to Print bottom 5 rows

"""
              City Colors Reported Shape Reported State              Time
18236   Grant Park             NaN       TRIANGLE    IL  12/31/2000 23:00
18237  Spirit Lake             NaN           DISK    IA  12/31/2000 23:00
18238  Eagle River             NaN            NaN    WI  12/31/2000 23:45
18239  Eagle River             RED          LIGHT    WI  12/31/2000 23:45
18240         Ybor             NaN           OVAL    FL  12/31/2000 23:59
"""


# To filter  rows()

# print(df.loc[6])

"""
City                  Crater Lake
Colors Reported               NaN
Shape Reported             CIRCLE
State                          CA
Time               6/15/1935 0:00
Name: 6, dtype: object
"""

print(df.loc[0:2])

"""
          City Colors Reported Shape Reported State             Time
0       Ithaca             NaN       TRIANGLE    NY   6/1/1930 22:00
1  Willingboro             NaN          OTHER    NJ  6/30/1930 20:00
2      Holyoke             NaN           OVAL    CO  2/15/1931 14:00

"""

print(df.loc[0:2, :])  # same output as above


print(df.loc[0:2 , 'City':'State']) # To filter rows and display columns from city to state Time column dropped
"""
          City Colors Reported Shape Reported State
0       Ithaca             NaN       TRIANGLE    NY
1  Willingboro             NaN          OTHER    NJ
2      Holyoke             NaN           OVAL    CO
"""

print(df.loc[: , 'City':'State'])  # To print all rows
"""
                       City Colors Reported Shape Reported State
0                    Ithaca             NaN       TRIANGLE    NY
1               Willingboro             NaN          OTHER    NJ
2                   Holyoke             NaN           OVAL    CO
3                   Abilene             NaN           DISK    KS
4      New York Worlds Fair             NaN          LIGHT    NY
...                     ...             ...            ...   ...
18236            Grant Park             NaN       TRIANGLE    IL
18237           Spirit Lake             NaN           DISK    IA
18238           Eagle River             NaN            NaN    WI
18239           Eagle River             RED          LIGHT    WI
18240                  Ybor             NaN           OVAL    FL

[18241 rows x 4 columns]

"""
print(df.head(3).drop('Time', axis = 1))

"""
Name: City, Length: 18241, dtype: object
          City Colors Reported Shape Reported State
0       Ithaca             NaN       TRIANGLE    NY
1  Willingboro             NaN          OTHER    NJ
2      Holyoke             NaN           OVAL    CO
"""
print(df.loc[: , 'City':'State'])  # To print all rows drop timecolumn

print(df.loc[: , 'City'])  # To print all rows and city column only

"""
0                      Ithaca
1                 Willingboro
2                     Holyoke
3                     Abilene
4        New York Worlds Fair
                 ...         
18236              Grant Park
18237             Spirit Lake
18238             Eagle River
18239             Eagle River
18240                    Ybor
Name: City, Length: 18241, dtype: object

"""

print(df[df.City == 'Abilene'])  # select Filtering by value

"""
          City Colors Reported Shape Reported State              Time
3      Abilene             NaN           DISK    KS    6/1/1931 13:00
6654   Abilene             NaN       TRIANGLE    TX     9/1/1991 1:00
8357   Abilene             NaN         SPHERE    TX    7/15/1995 0:00
8783   Abilene             NaN            NaN    KS  10/14/1995 23:20
10883  Abilene             NaN            NaN    TX  10/19/1997 20:45

"""

print(df[df.City == 'Abilene'].State) # Filter by value, select column

"""
3        KS
6654     TX
8357     TX
8783     KS
10883    TX
"""

print(df.columns)
# Index(['City', 'Colors Reported', 'Shape Reported', 'State', 'Time'], dtype='object')

print(df.iloc[0:3 , 0:3]) #Filter 3 rows and 3 columns

"""         City Colors Reported Shape Reported
0       Ithaca             NaN       TRIANGLE
1  Willingboro             NaN          OTHER
2      Holyoke             NaN           OVAL

"""

print(df.iloc[0:3 , :]) #Filter 3 rows and all columns
"""
          City Colors Reported Shape Reported State             Time
0       Ithaca             NaN       TRIANGLE    NY   6/1/1930 22:00
1  Willingboro             NaN          OTHER    NJ  6/30/1930 20:00
2      Holyoke             NaN           OVAL    CO  2/15/1931 14:00

"""

print(df.loc[:, ['City','Time']])  # to filter only two columns for all rows

"""
                       City              Time
0                    Ithaca    6/1/1930 22:00
1               Willingboro   6/30/1930 20:00
2                   Holyoke   2/15/1931 14:00
3                   Abilene    6/1/1931 13:00
4      New York Worlds Fair   4/18/1933 19:00
...                     ...               ...
18236            Grant Park  12/31/2000 23:00
18237           Spirit Lake  12/31/2000 23:00
18238           Eagle River  12/31/2000 23:45
18239           Eagle River  12/31/2000 23:45
18240                  Ybor  12/31/2000 23:59

[18241 rows x 2 columns]
"""

df1 = pd.read_csv('http://bit.ly/uforeports', index_col='City')

print(df1.head(5))

"""
                     Colors Reported Shape Reported State             Time
City                                                                      
Ithaca                           NaN       TRIANGLE    NY   6/1/1930 22:00
Willingboro                      NaN          OTHER    NJ  6/30/1930 20:00
Holyoke                          NaN           OVAL    CO  2/15/1931 14:00
Abilene                          NaN           DISK    KS   6/1/1931 13:00
New York Worlds Fair             NaN          LIGHT    NY  4/18/1933 19:00

"""

 Happy Learning @AMET!!!