主页 / user-5363686

Serge de Gosson de Varennes's questions

Martin Hope
Serge de Gosson de Varennes
Asked: 2024-02-20 21:52:16 +0800 CST
  • 5

我不完全确定这是否是这个问题的正确论坛（标签），所以如果它是错误的，我很抱歉。我面临的问题如下：

我正在创建一份报告，并将为许多客户进行分类。在其中，我有许多有关产品、可视化、表格等的统计数据。这些统计数据是使用BigQueryGCP 中的连接并使用python. 在最好的情况下，我会用 at tag 标记所有计算和数据检索单元hide-input，使用 jupyter nbconve 并创建 pdf 报告。不幸的是，由于我的环境限制，这不起作用。

但是，我可以将我的笔记本转换为 pdf*.tex文件，然后将该文件转换为 pdf 文件。但是，在这种情况下，所有单元格都是可见的。

有谁知道如何tex在转换为 pdf 期间隐藏文件的内容？

python
Martin Hope
Serge de Gosson de Varennes
Asked: 2024-02-06 01:27:54 +0800 CST
  • 5

假设我有一个 gcs 存储桶，其中包含具有以下结构的 json 文件：

[
  {
    "Id": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
    "Name": "alibaba",
    "storeid": "Y1",
    "storeName": "alibaba1",
    "a": "1/2/3",
    "b": "1.0/1.0/3",
    "c": "0/0/0",
    "d": "0/0/0",
    "e": "1.8/3.4",
    "f": "1/2/3",
    "g": "1/2/3",
  },
  {
    "Id": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
    "Name": "alibaba",
    "storeUuid": "Y2",
     "storeName": "alibaba2",
    "a": "1/2/3",
    "b": "1.0/1.0/3",
    "c": "0/0/0",
    "d": "0/0/0",
    "e": "1.7/2.4",
    "f": "1/2/3",
    "g": "1/2/3",
  },
  {
    "Id": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
    "Name": "alibaba",
    "storeUuid": "Y3",
     "storeName": "alibaba3",
    "a": "1/2/3",
    "b": "1.0/1.0/3",
    "c": "0/0/0",
    "d": "0/0/0",
    "e": "2.7/4.4",
    "f": "1/2/3",
    "g": "1/2/3",
  },
  {
    "Id": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
    "Name": "alibaba",
    "storeUuid": "Y4",
     "storeName": "alibaba4",
    "a": "1/2/3",
    "b": "1.0/1.0/3",
    "c": "0/0/0",
    "d": "0/0/0",
    "e": "3.7/5.4",
    "f": "1/2/3",
    "g": "1/2/3",
  }
]

我想要做的是通过求和a, b,c, d, f,g并取平均值来聚合不同的值e以返回一个json像

[
{
    "Id": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
    "Name": "alibaba",
    "a": "sum over all first instance/sum over all second instances/sum aover all third instance",
    "b": "sum over all first instance/sum over all second instances/sum aover all third instance",
    "c": "sum over all first instance/sum over all second instances/sum aover all third instance",
    "d": "sum over all first instance/sum over all second instances/sum aover all third instance",
    "e": "average over all first instance/average over all second instance",
    "f": "sum over all first instance/sum over all second instances/sum aover all third instance",
    "g": "sum over all first instance/sum over all second instances/sum aover all third instance",
  }
]

并不是说 中 的任何值都*/*/*可以是 NaN ，并且 中 的数据都e可以是 string data unvavailable。

在已经创建了这个函数

def format_large_numbers_optimized(value):
    abs_values = np.abs(value)
    mask = abs_values >= 1e6
    formatted_values = np.where(mask, 
                                np.char.add(np.round(value / 1e6, 2).astype(str), "M"), 
                                np.round(value, 2).astype(str))
    return formatted_values

def process_json_data_optimized(json_list):
    result = {}
    keys = set(json_list[0].keys()) - {'Id', 'Name', 'storeid', 'storeName'}
    for key in keys:
        result[key] = {'values': []}
    for json_data in json_list:
        for key in keys:
            value = json_data.get(key, '0')  
            result[key]['values'].append(value)
    for key in keys:
        all_values_processed = []
        for value in result[key]['values']:
            if isinstance(value, str) and '/' in value:
                processed_values = [float(v) if v != 'data unavailable' else 0 for v in value.split('/')]
            elif isinstance(value, float) or isinstance(value, int):
                processed_values = [value]
            else:
                processed_values = [0.0]  
            all_values_processed.append(processed_values)
        numeric_values = np.array(all_values_processed)
        if numeric_values.ndim == 1:
            numeric_values = numeric_values[:, np.newaxis]
        summed_values = np.sum(numeric_values, axis=0)
        formatted_summed_values = '/'.join(format_large_numbers_optimized(summed_values))
        result[key]['summed'] = formatted_summed_values
    processed_result = {key: data['summed'] for key, data in result.items()}
    processed_result['Id'] = json_list[0]['Id']
    processed_result['Name'] = json_list[0]['Name']
    return processed_result

但它并没有创造出我所期望的。我完全不知所措。非常感谢任何帮助。

python
Martin Hope
Serge de Gosson de Varennes
Asked: 2023-12-05 21:01:54 +0800 CST
  • 6

假设我有一个像这样的数据框

import pandas as pd

data = {
    'id1': [1]*12 + [1]*12 + [2]*12 + [2]*12,
    'id2': ['a']*12 + ['b']*12 + ['c']*12 + ['d']*12,
    'date': (['2023-11-20', '2023-11-21', '2023-11-22', '2023-11-23', '2023-11-24', '2023-11-25', '2023-11-26', '2023-11-27', '2023-11-28', '2023-11-29', '2023-11-30', '2023-12-01']*4)[:48],
    'event1': [10, 23, 36, 49, 62, 75, 88, 101, 114, 127, 140, 153, 12, 25, 38, 51, 64, 77, 90, 103, 116, 129, 142, 155, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 567, 785, 1003, 1221, 1439, 1657, 1875, 2093, 2311, 2529, 2747, 2965],
    'event2': ([0]*12 + [3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25] + [45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 95, 100, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]*2)[:48]
}

df = pd.DataFrame(data)

print(df)

正如您所看到的，对于每个 和的id2值都是累积的。也就是说，两个连续日期之间的值差是两个日期中较大日期的事件计数。event1event2

我想要为每个id2计算过去一天（此处为 2023-11-30）的事件数、过去三天的事件总数和过去 7 天的事件总数。

我尝试过以下类型的东西：

df.set_index('date', inplace=True)

target_date = pd.to_datetime('2023-11-30')

events_on_target_date = df.loc[target_date, 'event1'] - df.loc[target_date - pd.DateOffset(days=1), 'event1']
events_3_days_before = df.loc[target_date - pd.DateOffset(days=3):target_date, 'event1'].iloc[-1] - df.loc[target_date - pd.DateOffset(days=4), 'event1']
events_5_days_before = df.loc[target_date - pd.DateOffset(days=5):target_date, 'event1'].iloc[-1] - df.loc[target_date - pd.DateOffset(days=6), 'event1']

print(f"Number of events on {target_date}: {events_on_target_date}")
print(f"Total number of events in the 3 days before {target_date}: {events_3_days_before}")
print(f"Total number of events in the 5 days before {target_date}: {events_5_days_before}")

它适用于一个独立的 id1 和 id2，但我遇到了困难：

  • 将其应用到每个id2
  • 应用此包括计算event1和event2

期望的结果应该是这样的

id1 id2 count_event1_yesterday  count_event1_past3Days  count_event1_past5Days  count_event2_yesterday  count_event2_past3Days  count_event2_past5Days
1   a   13  39  52  0   0   0
1   b   13  39  52  4   6   8
2   c                       
2   d
pandas