Pandas的学习
·
-
一、Pandas 基础信息
- 介绍：Pandas 是一个开源的 Python 类库，主要用于数据分析、数据处理和数据可视化，具有高性能、数据结构易用、数据分析工具便捷等特点。
- 安装与导入：
- 安装方式：可通过pip install pandas或conda install pandas安装，推荐使用清华源（Simple Index）提高下载速度。
- 导入方式：通常使用import pandas as pd导入。
-
二、Pandas 核心数据结构
- Series：
- 性质：类似表格中的一列，属于一维数组，可保存任何数据类型，由索引和数据组成。
- 构造函数：pandas.Series(data, index, dtype, name, copy)，其中data为 ndarray 类型的一组数据；index为数据索引标签，默认从 0 开始；dtype为数据类型，默认自动判断；name用于设置名称；copy用于指定是否拷贝数据，默认为 False。
- 创建方式：
- 用列表创建，如arr=[1, 3, 6, 33, 1]，s1=pd.Series(arr)，不指定索引则默认从 0 开始。
- 用 ndarray 创建，如先通过n=np.random.randn(5)创建随机数组，再指定索引index=['a', 'b','c','d','e']，最后s2=pd.Series(n, index=index)。
- 用字典创建，如score_dict ={'zhangsan': 80, 'lisi':62, 'wangwu':97}，score=pd.Series(score_dict)。
- 查看索引和数值：通过score.index查看索引，score.values查看数值。
- DataFrame：
- 性质：表格型数据结构，含一组有序的列，每列可是不同值类型，既有行索引也有列索引。
- 构造函数：pandas.DataFrame(data, index, columns, dtype, copy)，data可为 ndarray、series、map、lists、dict 等类型；index为行标签；columns为列标签，默认是 RangeIndex (0, 1, 2, …, n)；dtype为数据类型；copy默认为 False。
- 创建方式：
- 用列表创建，如data =[['zhangsan', 10], ['lisi', 12], ['wangwu', 13]]，df = pd.DataFrame(data, columns=['site', 'Age'], dtype=float)。
- 用 ndarrays 创建，如data ={'site': ['zhangsan', 'lisi', 'wangwu'], 'Age':[10, 12, 13]}，df = pd.DataFrame(data)。
- 用字典（key/value）创建，如data = [{'a': 1, 'b': 2}, {'a':5, 'b': 10, 'c': 20}]，df = pd.DataFrame(data)。
-
三、Pandas 数据查询
- loc () 方法：用于返回指定行数据。
- 返回单行：如df.loc[0]返回第一行，df.loc[1]返回第二行。
- 返回多行：使用[[...]]格式，如df.loc[[0, 1]]返回第一行和第二行。
- 指定索引查询：先在创建 DataFrame 时指定索引，如df = pd.DataFrame(data, index =["Chinese", "Math", "English"])，再用df.loc["Chinese"]返回该索引对应的行。
-
四、Pandas 数据读取
- 不同类型文件读取方法：
- csv、tsv、txt 文件：用pd.read_csv()读取。
- excel 文件（xls 或 xlsx）：用pd.read_excel()读取。
- mysql 关系型数据库表：用pd.read_sql()读取。
- 查看数据的方法：
- head()：用于读取前面的 n 行，默认返回 5 行，如df.head()返回前 5 行，df.head(3)返回前 3 行。
- tail()：用于读取尾部的 n 行，默认返回 5 行，如df.tail()返回最后 5 行，df.tail(3)返回最后 3 行。
- info()：返回表格的基本信息，包括数据类型、非空值数量、内存占用等。
-
五、Pandas 处理丢失数据
- 判断空值：使用isnull()方法判断各个单元格是否为空，如df['NUM_BEDROOMS'].isnull()。
- 自定义缺失值标识：读取文件时，可通过na_values指定需识别为缺失值的字符串，如missing_values = ["n/a", "na","-"]，df = pd.read_csv('property-data.csv', na_values = missing_values)。
- 删除含空值的行：使用dropna()方法，语法为DataFrame.dropna(axis=0, how='any', thresh=None, subset=None, inplace=False)。其中axis默认为 0（剔除整行）；how为 'any' 时，有一个空值就删行，为 'all' 时，所有数据为空才删行；thresh设置保留所需的非空值数量；subset设置要检查的列；inplace为 True 时直接修改源数据。
- 填充空值：
- 用指定值替换：通过fillna()方法，如df.fillna(666, inplace = True)用 666 替换空值。
- 用均值替换：先计算列的均值，再填充，如x = df["ST_NUM"].mean()，df["ST_NUM"].fillna(x, inplace = True)。
- 用中位数替换：先计算列的中位数，再填充，如x = df["ST_NUM"].median()，df["ST_NUM"].fillna(x, inplace = True)。
-
#代码展示： import pandas as pd arr = [1, 3, 6, 33, 1] s1 = pd.Series(arr) # 如果不指定索引，则默认从 0 开始 print(s1) #结果： 0 1 1 3 2 6 3 33 4 1 dtype: int64 import numpy as np import pandas as pd n = np.random.randn(5) # 创建一个随机 Ndarray 数组 index = ['a', 'b', 'c', 'd', 'e'] s2 = pd.Series(n, index = index) print(s2) #结果： a 1.376382 b -0.667707 c 0.364362 d -0.572639 e -0.927510 dtype: float64 import csv with open('./iris.csv','r') as csvfile: read = csv.reader(csvfile) for row in read: print(row) #结果： ['', 'Sepal.Length', 'Sepal.Width', 'Petal.Length', 'Petal.Width', 'Species'] ['1', '5.1', '3.5', '1.4', '0.2', 'setosa'] ['2', '4.9', '3', '1.4', '0.2', 'setosa'] ['3', '4.7', '3.2', '1.3', '0.2', 'setosa'] ['4', '4.6', '3.1', '1.5', '0.2', 'setosa'] ['5', '5', '3.6', '1.4', '0.2', 'setosa'] ['6', '5.4', '3.9', '1.7', '0.4', 'setosa'] ['7', '4.6', '3.4', '1.4', '0.3', 'setosa'] ['8', '5', '3.4', '1.5', '0.2', 'setosa'] ['9', '4.4', '2.9', '1.4', '0.2', 'setosa'] ['10', '4.9', '3.1', '1.5', '0.1', 'setosa'] ['11', '5.4', '3.7', '1.5', '0.2', 'setosa'] ['12', '4.8', '3.4', '1.6', '0.2', 'setosa'] ['13', '4.8', '3', '1.4', '0.1', 'setosa'] ['14', '4.3', '3', '1.1', '0.1', 'setosa'] ['15', '5.8', '4', '1.2', '0.2', 'setosa'] ['16', '5.7', '4.4', '1.5', '0.4', 'setosa'] ['17', '5.4', '3.9', '1.3', '0.4', 'setosa'] ['18', '5.1', '3.5', '1.4', '0.3', 'setosa'] ['19', '5.7', '3.8', '1.7', '0.3', 'setosa'] ['20', '5.1', '3.8', '1.5', '0.3', 'setosa'] ['21', '5.4', '3.4', '1.7', '0.2', 'setosa'] ['22', '5.1', '3.7', '1.5', '0.4', 'setosa'] ['23', '4.6', '3.6', '1', '0.2', 'setosa'] ['24', '5.1', '3.3', '1.7', '0.5', 'setosa'] ['25', '4.8', '3.4', '1.9', '0.2', 'setosa'] ['26', '5', '3', '1.6', '0.2', 'setosa'] ['27', '5', '3.4', '1.6', '0.4', 'setosa'] ['28', '5.2', '3.5', '1.5', '0.2', 'setosa'] ['29', '5.2', '3.4', '1.4', '0.2', 'setosa'] ['30', '4.7', '3.2', '1.6', '0.2', 'setosa'] ['31', '4.8', '3.1', '1.6', '0.2', 'setosa'] ['32', '5.4', '3.4', '1.5', '0.4', 'setosa'] ['33', '5.2', '4.1', '1.5', '0.1', 'setosa'] ['34', '5.5', '4.2', '1.4', '0.2', 'setosa'] ['35', '4.9', '3.1', '1.5', '0.2', 'setosa'] ['36', '5', '3.2', '1.2', '0.2', 'setosa'] ['37', '5.5', '3.5', '1.3', '0.2', 'setosa'] ['38', '4.9', '3.6', '1.4', '0.1', 'setosa'] ['39', '4.4', '3', '1.3', '0.2', 'setosa'] ['40', '5.1', '3.4', '1.5', '0.2', 'setosa'] ['41', '5', '3.5', '1.3', '0.3', 'setosa'] ['42', '4.5', '2.3', '1.3', '0.3', 'setosa'] ['43', '4.4', '3.2', '1.3', '0.2', 'setosa'] ['44', '5', '3.5', '1.6', '0.6', 'setosa'] ['45', '5.1', '3.8', '1.9', '0.4', 'setosa'] ['46', '4.8', '3', '1.4', '0.3', 'setosa'] ['47', '5.1', '3.8', '1.6', '0.2', 'setosa'] ['48', '4.6', '3.2', '1.4', '0.2', 'setosa'] ['49', '5.3', '3.7', '1.5', '0.2', 'setosa'] ['50', '5', '3.3', '1.4', '0.2', 'setosa'] ['51', '7', '3.2', '4.7', '1.4', 'versicolor'] ['52', '6.4', '3.2', '4.5', '1.5', 'versicolor'] ['53', '6.9', '3.1', '4.9', '1.5', 'versicolor'] ['54', '5.5', '2.3', '4', '1.3', 'versicolor'] ['55', '6.5', '2.8', '4.6', '1.5', 'versicolor'] ['56', '5.7', '2.8', '4.5', '1.3', 'versicolor'] ['57', '6.3', '3.3', '4.7', '1.6', 'versicolor'] ['58', '4.9', '2.4', '3.3', '1', 'versicolor'] ['59', '6.6', '2.9', '4.6', '1.3', 'versicolor'] ['60', '5.2', '2.7', '3.9', '1.4', 'versicolor'] ['61', '5', '2', '3.5', '1', 'versicolor'] ['62', '5.9', '3', '4.2', '1.5', 'versicolor'] ['63', '6', '2.2', '4', '1', 'versicolor'] ['64', '6.1', '2.9', '4.7', '1.4', 'versicolor'] ['65', '5.6', '2.9', '3.6', '1.3', 'versicolor'] ['66', '6.7', '3.1', '4.4', '1.4', 'versicolor'] ['67', '5.6', '3', '4.5', '1.5', 'versicolor'] ['68', '5.8', '2.7', '4.1', '1', 'versicolor'] ['69', '6.2', '2.2', '4.5', '1.5', 'versicolor'] ['70', '5.6', '2.5', '3.9', '1.1', 'versicolor'] ['71', '5.9', '3.2', '4.8', '1.8', 'versicolor'] ['72', '6.1', '2.8', '4', '1.3', 'versicolor'] ['73', '6.3', '2.5', '4.9', '1.5', 'versicolor'] ['74', '6.1', '2.8', '4.7', '1.2', 'versicolor'] ['75', '6.4', '2.9', '4.3', '1.3', 'versicolor'] ['76', '6.6', '3', '4.4', '1.4', 'versicolor'] ['77', '6.8', '2.8', '4.8', '1.4', 'versicolor'] ['78', '6.7', '3', '5', '1.7', 'versicolor'] ['79', '6', '2.9', '4.5', '1.5', 'versicolor'] ['80', '5.7', '2.6', '3.5', '1', 'versicolor'] ['81', '5.5', '2.4', '3.8', '1.1', 'versicolor'] ['82', '5.5', '2.4', '3.7', '1', 'versicolor'] ['83', '5.8', '2.7', '3.9', '1.2', 'versicolor'] ['84', '6', '2.7', '5.1', '1.6', 'versicolor'] ['85', '5.4', '3', '4.5', '1.5', 'versicolor'] ['86', '6', '3.4', '4.5', '1.6', 'versicolor'] ['87', '6.7', '3.1', '4.7', '1.5', 'versicolor'] ['88', '6.3', '2.3', '4.4', '1.3', 'versicolor'] ['89', '5.6', '3', '4.1', '1.3', 'versicolor'] ['90', '5.5', '2.5', '4', '1.3', 'versicolor'] ['91', '5.5', '2.6', '4.4', '1.2', 'versicolor'] ['92', '6.1', '3', '4.6', '1.4', 'versicolor'] ['93', '5.8', '2.6', '4', '1.2', 'versicolor'] ['94', '5', '2.3', '3.3', '1', 'versicolor'] ['95', '5.6', '2.7', '4.2', '1.3', 'versicolor'] ['96', '5.7', '3', '4.2', '1.2', 'versicolor'] ['97', '5.7', '2.9', '4.2', '1.3', 'versicolor'] ['98', '6.2', '2.9', '4.3', '1.3', 'versicolor'] ['99', '5.1', '2.5', '3', '1.1', 'versicolor'] ['100', '5.7', '2.8', '4.1', '1.3', 'versicolor'] ['101', '6.3', '3.3', '6', '2.5', 'virginica'] ['102', '5.8', '2.7', '5.1', '1.9', 'virginica'] ['103', '7.1', '3', '5.9', '2.1', 'virginica'] ['104', '6.3', '2.9', '5.6', '1.8', 'virginica'] ['105', '6.5', '3', '5.8', '2.2', 'virginica'] ['106', '7.6', '3', '6.6', '2.1', 'virginica'] ['107', '4.9', '2.5', '4.5', '1.7', 'virginica'] ['108', '7.3', '2.9', '6.3', '1.8', 'virginica'] ['109', '6.7', '2.5', '5.8', '1.8', 'virginica'] ['110', '7.2', '3.6', '6.1', '2.5', 'virginica'] ['111', '6.5', '3.2', '5.1', '2', 'virginica'] ['112', '6.4', '2.7', '5.3', '1.9', 'virginica'] ['113', '6.8', '3', '5.5', '2.1', 'virginica'] ['114', '5.7', '2.5', '5', '2', 'virginica'] ['115', '5.8', '2.8', '5.1', '2.4', 'virginica'] ['116', '6.4', '3.2', '5.3', '2.3', 'virginica'] ['117', '6.5', '3', '5.5', '1.8', 'virginica'] ['118', '7.7', '3.8', '6.7', '2.2', 'virginica'] ['119', '7.7', '2.6', '6.9', '2.3', 'virginica'] ['120', '6', '2.2', '5', '1.5', 'virginica'] ['121', '6.9', '3.2', '5.7', '2.3', 'virginica'] ['122', '5.6', '2.8', '4.9', '2', 'virginica'] ['123', '7.7', '2.8', '6.7', '2', 'virginica'] ['124', '6.3', '2.7', '4.9', '1.8', 'virginica'] ['125', '6.7', '3.3', '5.7', '2.1', 'virginica'] ['126', '7.2', '3.2', '6', '1.8', 'virginica'] ['127', '6.2', '2.8', '4.8', '1.8', 'virginica'] ['128', '6.1', '3', '4.9', '1.8', 'virginica'] ['129', '6.4', '2.8', '5.6', '2.1', 'virginica'] ['130', '7.2', '3', '5.8', '1.6', 'virginica'] ['131', '7.4', '2.8', '6.1', '1.9', 'virginica'] ['132', '7.9', '3.8', '6.4', '2', 'virginica'] ['133', '6.4', '2.8', '5.6', '2.2', 'virginica'] ['134', '6.3', '2.8', '5.1', '1.5', 'virginica'] ['135', '6.1', '2.6', '5.6', '1.4', 'virginica'] ['136', '7.7', '3', '6.1', '2.3', 'virginica'] ['137', '6.3', '3.4', '5.6', '2.4', 'virginica'] ['138', '6.4', '3.1', '5.5', '1.8', 'virginica'] ['139', '6', '3', '4.8', '1.8', 'virginica'] ['140', '6.9', '3.1', '5.4', '2.1', 'virginica'] ['141', '6.7', '3.1', '5.6', '2.4', 'virginica'] ['142', '6.9', '3.1', '5.1', '2.3', 'virginica'] ['143', '5.8', '2.7', '5.1', '1.9', 'virginica'] ['144', '6.8', '3.2', '5.9', '2.3', 'virginica'] ['145', '6.7', '3.3', '5.7', '2.5', 'virginica'] ['146', '6.7', '3', '5.2', '2.3', 'virginica'] ['147', '6.3', '2.5', '5', '1.9', 'virginica'] ['148', '6.5', '3', '5.2', '2', 'virginica'] ['149', '6.2', '3.4', '5.4', '2.3', 'virginica'] ['150', '5.9', '3', '5.1', '1.8', 'virginica'] import numpy as np import pandas as pd score_dict = {'zhangsan': 80, 'lisi': 62, 'wangwu': 97} score = pd.Series(score_dict) print(score) #结果： zhangsan 80 lisi 62 wangwu 97 dtype: int64 import numpy as np import pandas as pd score_dict = {'zhangsan': 80, 'lisi': 62, 'wangwu': 97} score = pd.Series(score_dict) print(score) #结果： zhangsan 80 lisi 62 wangwu 97 dtype: int64 import pandas as pd data = {'Site':['zhangsan', 'lisi', 'wangwu'], 'Age':[10, 12, 13]} df = pd.DataFrame(data) print(df) #结果： Site Age 0 zhangsan 10 1 lisi 12 2 wangwu 13 import pandas as pd data = [{'a': 1, 'b': 2}, {'a': 5, 'b': 10, 'c': 20}] df = pd.DataFrame(data) print(df) #结果: a b c 0 1 2 NaN 1 5 10 20.0 import pandas as pd data = { "zhangsan": [99, 60, 70], "lisi": [80, 70, 95] } df = pd.DataFrame(data) print(df.loc[0]) print(df.loc[1]) #结果： zhangsan 99 lisi 80 Name: 0, dtype: int64 zhangsan 60 lisi 70 Name: 1, dtype: int64 import pandas as pd data = { "zhangsan": [99, 60, 70], "lisi": [80, 70, 95] } df = pd.DataFrame(data) print(df.loc[[0, 1]]) #结果： zhangsan lisi 0 99 80 1 60 70 import pandas as pd data = { "zhangsan": [99, 60, 70], "lisi": [80, 70, 95] } df = pd.DataFrame(data, index = ["Chinese", "Math", "English"]) print(df) #结果： zhangsan lisi Chinese 99 80 Math 60 70 English 70 95 import pandas as pd data = { "zhangsan": [99, 60, 70], "lisi": [80, 70, 95] } df = pd.DataFrame(data, index = ["Chinese", "Math", "English"]) print(df.loc["Chinese"]) #结果： zhangsan 99 lisi 80 Name: Chinese, dtype: int64 import pandas as pd df = pd.read_csv('nba.csv') print(df.head()) #结果： Name Team Number ... Weight College Salary 0 Avery Bradley Boston Celtics 0.0 ... 180.0 Texas 7730337.0 1 Jae Crowder Boston Celtics 99.0 ... 235.0 Marquette 6796117.0 2 John Holland Boston Celtics 30.0 ... 205.0 Boston University NaN 3 R.J. Hunter Boston Celtics 28.0 ... 185.0 Georgia State 1148640.0 4 Jonas Jerebko Boston Celtics 8.0 ... 231.0 NaN 5000000.0 [5 rows x 9 columns] import pandas as pd df = pd.read_csv('nba.csv') print(df.head(3)) #结果： Name Team Number ... Weight College Salary 0 Avery Bradley Boston Celtics 0.0 ... 180.0 Texas 7730337.0 1 Jae Crowder Boston Celtics 99.0 ... 235.0 Marquette 6796117.0 2 John Holland Boston Celtics 30.0 ... 205.0 Boston University NaN [3 rows x 9 columns]
有“AI”的1024 = 2048，欢迎大家加入2048 AI社区
更多推荐
2
0- 0
已为社区贡献1条内容
所有评论(0)