此文章是vip文章,如何查看?  

1,点击链接获取密钥 http://nicethemes.cn/product/view29882.html

2,在下方输入文章查看密钥即可立即查看当前vip文章


60 pandas 时间序列-详述重新采样resample(tcy)

  • 时间:
  • 浏览:
  • 来源:互联网

重新采样 2019/1/21

resample()是一个基于时间的groupby,然后是每个组的缩减方法。resample功能非常灵活,允许您指定许多不同的参数来控制频率转换和重采样操作

1.函数 

df.resample(rule,how=None,axis=0,fill_method=None,closed=None,
label=None,convention=start, kind=None, loffset=None,
limit=None, base=0, on=None, level=None) 
用途:频率转换和重新采样时间的便捷方法
    # 对象必须具有类似日期时间的索引(DatetimeIndex,PeriodIndex或TimedeltaIndex)
    # 或传递datetime值到on或level关键字
返回:Resampler object 

参数: 

rule : str#目标转换的偏移字符串或对象
axis=0:int
closed ='left': {'right', 'left'}#bin间隔的哪一侧是关闭的
# 除M,A,Q,BM,BA,BQ,W(默认右闭合)之外的所有频率偏移
label='left':{'right','left'}#哪个bin边缘标签用于标记桶bucket
# 除M,A,Q,BM,BA,BQ,W(默认右闭合)
convention : {'start', 'end', 's', 'e'}#仅适用PeriodIndex是使用开头还是结尾
kind:{'timestamp','period'}#结果转换为时间戳索引或时间段索引;默认保留输入表示
loffset:timedelta#调整重新采样的时间标签
base=0:int#对于均匀细分1天的频率,其中的“origin”聚合间隔。
# 例如,对于“5分钟”频率,基数可以范围从0到4.默认为0
on : str#df要使用的列而不是索引进行重新采样。列必须与日期时间相似。
level:str或int#对MultiIndex用级别(名称或编号)重采样。级别必须是日期

2.频率转换

index = pd.date_range('1/21/2019', periods=9, freq='T')
s= pd.Series(range(9), index=index)
s
'''
2019-01-21 00:00:00 0
2019-01-21 00:01:00 1
2019-01-21 00:02:00 2
2019-01-21 00:03:00 3
2019-01-21 00:04:00 4
2019-01-21 00:05:00 5
2019-01-21 00:06:00 6
2019-01-21 00:07:00 7
2019-01-21 00:08:00 8
Freq: T, dtype: int64
'''

实例: 

s.resample('3S').asfreq()

3.1.上采样

实例1.1:上采样Upsample 

s.resample('30S').asfreq()[0:5]
'''
2019-01-21 00:00:00 0.0
2019-01-21 00:00:30 NaN
2019-01-21 00:01:00 1.0
2019-01-21 00:01:30 NaN
2019-01-21 00:02:00 2.0
Freq: 30S, dtype: float64
''' 

实例1.2:上采样Upsample-填充NaN

s[:2].resample('250L').ffill(limit=2)
s.resample('30S').pad()[0:5]  #用前面值填充后面Na
s.resample('30S').bfill()[0:5]#用后面值填充前面Na 

3.2下采样

实例2.1:下采样

s.resample('2min').asfreq()
'''
2019-01-21 00:00:00    0
2019-01-21 00:02:00    2
2019-01-21 00:04:00    4
2019-01-21 00:06:00    6
2019-01-21 00:08:00    8
Freq: 2T, dtype: int64
''' 

实例2.2:对于下采样,closed可以设置为“左”或“右”以指定间隔的哪一端关闭:

s.resample('5Min', closed='left').sum()
'''
2019-01-21 00:00:00 10
2019-01-21 00:05:00 26
Freq: 5T, dtype: int64
'''
s.resample('5Min', closed='right').sum()
'''
2019-01-20 23:55:00 0
2019-01-21 00:00:00 15
2019-01-21 00:05:00 21
Freq: 5T, dtype: int64
''' 

4.1参数:label和loffset

参数labelloffset用于处理结果标签。label指定结果是使用间隔的开头还是结尾标记。loffset对输出标签执行时间调整。 

s.resample('5Min').sum()
s.resample('5Min',label='left').sum()
''' 
2019-01-21 00:00:00 10
2019-01-21 00:05:00 26
Freq: 5T, dtype: int64
'''
s.resample('5Min',label='left',loffset='1s').sum()
'''
2019-01-21 00:00:01 10
2019-01-21 00:05:01 26
dtype: int64
'''

注意 

缺省值 labelclosed =“left”;除“M”,“A”,“Q”,“BM”,“BA”,“BQ”,和“W”

4.2参数:label和closed

rng2 = pd.date_range('1/1/2019', end='3/31/2019', freq='D')
ts2 = pd.Series(range(len(rng2)), index=rng2)
ts2.resample('SM', label='right', closed='right').max().dropna()#推荐用

实例: 

ts2.resample('M',label='left', closed='left').max()
'''
2018-12-31 29 label='left'索引向前推进一个单位,本例中向前推进1天
2019-01-31 57 closed='left'数值不包含本月最后1天,57是2月27日的数值
2019-02-28 88
2019-03-31 89
Freq: M, dtype: int64
'''
ts2.resample('M',label='left', closed='right').max()
'''
2018-12-31 30 
2019-01-31 58 closed='right'数值包含本月第1天,58是2月28日月末的数值
2019-02-28 89
Freq: M, dtype: int64
'''
ts2.resample('M',label='right', closed='left').max()
'''
2019-01-31 29 label='right'索引为本月的最后一日即1月31日
2019-02-28 57
2019-03-31 88
2019-04-30 89
Freq: M, dtype: int64
'''
ts2.resample('M',label='right', closed='right').max()
ts2.resample('M').max()# default: label='right', closed='right'
'''
2019-01-31 30
2019-02-28 58
2019-03-31 89
Freq: M, dtype: int64
'''
# default: label='left', closed='left'
ts2.resample('SM').max()#半月
'''
2018-12-31 13
2019-01-15 29
2019-01-31 44
2019-02-15 57
2019-02-28 72
2019-03-15 88
2019-03-31 89
Freq: SM-15, dtype: int64
'''
ts2.resample('SM', label='right', closed='right').max()
'''
2019-01-15 14.0
2019-01-31 30.0
2019-02-15 45.0
2019-02-28 58.0
2019-03-15 73.0
2019-03-31 89.0
2019-04-15 NaN
Freq: SM-15, dtype: float64
'''

 

4.3.axis/kind参数

axis=0或1允许您重新采样指定轴
kind='timestamp'或'period',将结果索引转换为时间戳/时间范围表示。默认保留输入

4.4.convention参数

实例1.1:convention参数 下采样

重新采样周期数据时可以设置为“开始”或“结束”它指定低频周期如何转换为更高频率周期。

idx=pd.period_range('2019-01-21',freq='D',periods=90)
s = pd.Series(np.arange(90)+10,index=idx)#此种用法上采样显得多余
s1=pd.Series([1,2],pd.period_range('2019-01-21',freq='A',periods=2))
s
'''
2019-01-21 10
2019-01-22 11
..
2019-04-19 98
2019-04-20 99
Freq: D, Length: 90, dtype: int32
'''

实例1.2:

s[~s.asfreq('M').index.duplicated(keep='first')]#以下3个结果相同,重采样不允许重复索引,asfreq允许重复索引
s[~s.asfreq('M').index.duplicated(keep='first')].resample('M', convention='end').asfreq()
s[~s.asfreq('M').index.duplicated(keep='first')].resample('M', convention='start').asfreq()
'''
2019-01-21 10
2019-02-01 21
2019-03-01 49
2019-04-01 80
Freq: D, dtype: int32
'''

实例1.3:

s[~s.asfreq('M').index.duplicated(keep='last')]#以下3个结果相同,重采样不允许重复索引,asfreq允许重复索引
s[~s.asfreq('M').index.duplicated(keep='last')].resample('M', convention='start').asfreq()
s[~s.asfreq('M').index.duplicated(keep='last')].resample('M', convention='end').asfreq()
'''
2019-01-31 20
2019-02-28 48
2019-03-31 79
2019-04-20 99
Freq: D, dtype: int32
'''

实例2.1:convention参数 上采样

s1.resample('M', convention='start').asfreq().head(3)
'''
2019-01 1.0
2019-02 NaN
2019-03 NaN
Freq: M, dtype: float64
'''
s1.resample('M', convention='end').asfreq().head(3)
'''
2019-12 1.0
2020-01 NaN
2020-02 NaN
Freq: M, dtype: float64
'''

4.5.on参数

实例1:on参数=时间列/level=str or int

# 实例1:无时间索引,根据datetime列重取样将其传递给on关键字
rng=pd.date_range('2019-01-21',freq='W',periods=3)
idx=pd.MultiIndex.from_arrays([[-1,-2,-3],rng],names=['N1','N2'])
df1 = pd.DataFrame({'date':rng,'a': np.arange(11,14)},index=idx)
'''
date a
N1 N2
-1 2019-01-27 2019-01-27 11
-2 2019-02-03 2019-02-03 12
-3 2019-02-10 2019-02-10 13
'''
df1.resample('M', on='date').sum()
'''
a
date
2019-01-31 11
2019-02-28 25
''' 

实例2:按日期时间级别重新采样,则MultiIndex可以将其名称或位置传递给 level关键字

df1.resample('M',level='N2').sum()
df1.resample('M',level=1).sum()
'''
a
N2
2019-01-31 11
2019-02-28 25
''' 

5.重采样对象方法

重采样返回的对象的方法包括summeanstdsem, maxminmedianfirstlastohlc

实例1

s.resample('5Min').mean()
''' 
2019-01-21 00:00:00 2.0
2019-01-21 00:05:00 6.5
Freq: 5T, dtype: float64
'''
s.resample('5Min').ohlc()
'''
open high low close
2019-01-21 00:00:00 0 4 0 4
2019-01-21 00:05:00 5 8 5 8
'''
s.resample('5Min').max()
'''
2019-01-21 00:00:00 4
2019-01-21 00:05:00 8
Freq: 5T, dtype: int64
'''

实例2:

s.resample('3T').sum()
s.resample('3T', label='right').sum()
'''
2019-01-21 00:00:00     3   [0,3)=0+1+2
2019-01-21 00:03:00    12   [3,6)=3+4+5
2019-01-21 00:06:00    21   [6,9)=6+7+8
Freq: 3T, dtype: int64
''' 

实例3:

t=s.resample('3T', label='right', closed='right')
t.count()
'''
2019-01-21 00:00:00    1
2019-01-21 00:03:00    3
2019-01-21 00:06:00    3
2019-01-21 00:09:00    2
Freq: 3T, dtype: int64
'''
s.resample('3T', label='right', closed='right').sum()
'''
2019-01-21 00:00:00     0       =0
2019-01-21 00:03:00     6  (0,3]=1+2+3
2019-01-21 00:06:00    15  (3,6]=4+5+6
2019-01-21 00:09:00    15  (6,8]=7+8
Freq: 3T, dtype: int64
''' 

6.apply传递自定义函数

实例:

def custom_resampler(array):
return np.sum(array)+5

s.resample('3T').apply(custom_resampler)
'''
2019-01-21 00:00:00     8 [0,3)=0+1+2+5
2019-01-21 00:03:00    17 [3,6)=3+4+5+5
2019-01-21 00:06:00    26 [6,9)=6+7+8+5
Freq: 3T, dtype: int64
'''

7.agg

与聚合,groupby和窗口函数API类似,Resampler可以选择性地重采样:

实例5.1:重新采样方法:默认对所有列操作

idx=pd.date_range(pd.Timestamp.now(), freq='S', periods=9)
df = pd.DataFrame(np.arange(27).reshape(3,9).T+10,index=idx,columns=['A','B','C'])
df
'''
                             A   B   C
2019-01-21 13:57:37.155107  10  19  28
2019-01-21 13:57:38.155107  11  20  29
2019-01-21 13:57:39.155107  12  21  30
2019-01-21 13:57:40.155107  13  22  31
2019-01-21 13:57:41.155107  14  23  32
2019-01-21 13:57:42.155107  15  24  33
2019-01-21 13:57:43.155107  16  25  34
2019-01-21 13:57:44.155107  17  26  35
2019-01-21 13:57:45.155107  18  27  36
'''
r = df.resample('2S')
r.first()
'''
                      A   B   C
2019-01-21 13:57:36  10  19  28
2019-01-21 13:57:38  11  20  29
2019-01-21 13:57:40  13  22  31
2019-01-21 13:57:42  15  24  33
2019-01-21 13:57:44  17  26  35
'''
r.count()
'''
                     A  B  C
2019-01-21 13:57:36  1  1  1
2019-01-21 13:57:38  2  2  2
2019-01-21 13:57:40  2  2  2
2019-01-21 13:57:42  2  2  2
2019-01-21 13:57:44  2  2  2
'''

实例:

r['A'].sum() #用getitem选择一或多个列
r[['A','B']].sum() #用getitem选择一或多个列
r[['A','B','C']].sum()#用getitem选择一或多个列
r.sum() #等价上面1行
'''
A B C
2019-01-21 13:57:36 10 19 28 10=10(13:57:38)
2019-01-21 13:57:38 23 41 59 23=11(13:57:38)+12(13:57:39)
2019-01-21 13:57:40 27 45 63 27=13(13:57:40)+14(13:57:41)
2019-01-21 13:57:42 31 49 67 31=15(13:57:42)+16(13:57:43)
2019-01-21 13:57:44 35 53 71 35=17(13:57:44)+18(13:57:45)
''' 

实例2:df.resumple.A.agg([函数]或字典)聚合,输出DataFrame

r['A'].agg([np.sum, np.mean, np.std]).round(2)
'''
                     sum  mean   std
2019-01-21 13:57:36   10  10.0   NaN
2019-01-21 13:57:38   23  11.5  0.71
2019-01-21 13:57:40   27  13.5  0.71
2019-01-21 13:57:42   31  15.5  0.71
2019-01-21 13:57:44   35  17.5  0.71
'''

实例3:df.resample.agg([函数])聚合,生成分层索引结果:

r.agg([np.sum, np.mean]).round(2)
'''
A B C
sum mean sum mean sum mean
2019-01-21 13:57:36 10 10.0 19 19.0 28 28.0
2019-01-21 13:57:38 23 11.5 41 20.5 59 29.5
2019-01-21 13:57:40 27 13.5 45 22.5 63 31.5
2019-01-21 13:57:42 31 15.5 49 24.5 67 33.5
2019-01-21 13:57:44 35 17.5 53 26.5 71 35.5
''' 

实例4:df.resample.agg([dict])聚合

r.agg({'A':np.sum,'B':lambda x:np.std(x, ddof=1)}).round(2)
r.agg({'A': 'sum','B':'std'}).round(2)#函数名称也可以是字符串
'''
A B
2019-01-21 13:57:36 10 NaN
2019-01-21 13:57:38 23 0.71
2019-01-21 13:57:40 27 0.71
2019-01-21 13:57:42 31 0.71
2019-01-21 13:57:44 35 0.71
'''

实例5:分别为每列指定多个聚合函数 

r.agg({'A':['sum','std'],'B':['mean','std']}).round(2)
'''
A B
sum std mean std
2019-01-21 13:57:36 10 NaN 19.0 NaN
2019-01-21 13:57:38 23 0.71 20.5 0.71
2019-01-21 13:57:40 27 0.71 22.5 0.71
2019-01-21 13:57:42 31 0.71 24.5 0.71
2019-01-21 13:57:44 35 0.71 26.5 0.71
'''

6.稀疏重采样

稀疏时间序列相对于您要重新采样的时间量而言,点数要少得多。可能会产生大量中间值nan。

rng = pd.date_range(pd.Timestamp.now(), periods=100, freq='D') + pd.Timedelta('1s')
ts = pd.Series(range(100), index=rng)
ts
'''
2019-01-21 19:10:54.195179 0
2019-01-22 19:10:54.195179 1
..
2019-04-29 19:10:54.195179 98
2019-04-30 19:10:54.195179 99
Freq: D, Length: 100, dtype: int64
'''
ts.resample('3T').sum()
'''
2019-01-21 19:09:00 0
2019-01-21 19:12:00 0
2019-01-21 19:15:00 0
..
2019-04-30 19:09:00 99
Freq: 3T, Length: 47521, dtype: int64
'''

重新采样群组:

from functools import partial
from pandas.tseries.frequencies import to_offset
def round(t, freq):
freq = to_offset(freq)
return pd.Timestamp((t.value // freq.delta.value) * freq.delta.value)

ts.groupby(partial(round, freq='3T')).sum()
'''
2019-01-21 19:09:00 0
2019-01-22 19:09:00 1
2019-01-23 19:09:00 2
..
2019-04-30 19:09:00 99
Length: 100, dtype: int64
'''

本文链接http://element-ui.cn/news/show-461701.aspx