基于多突触发放神经元的推箱任务中异常物理现象与智能体认知分析

Anomalous Physical Phenomena and Agent Cognition Analysis in a Box-Pushing Task Using Multi-Synaptic Firing Neurons

---

摘要

本研究构建了基于多突触发放（MSF）神经元的脉冲神经网络智能体，控制四轮小车在平坦地面推动箱子。实验持续60秒，箱子从15.0米被推至16.24米，净位移1.24米，右队累计奖励高达239万，左队累计奖励为-331万。尽管智能体成功学会了将箱子向右推动，但推箱距离远小于奖励数值所暗示的“高效学习”，且仍存在箱子被抬升至机器人顶部的异常物理现象。本文从认知科学与认知心理学角度，结合智能体的感知-决策-动作循环、奖励调制STDP（RSTDP）学习机制和遗传算法（GA）演化，分析这些异常行为的成因，并探讨奖励结构、物理参数与学习策略之间的耦合关系。最终提出改进方向，以促进智能体在复杂物理环境中的稳定学习。

Abstract
This study constructs spiking neural network agents based on Multi-Synaptic Firing (MSF) neurons to control four-wheel cars pushing a box on flat ground. The experiment lasted 60 seconds, moving the box from 15.0 m to 16.24 m, a net displacement of 1.24 m. The right team accumulated a massive reward of 2.39 million, while the left team accumulated -3.32 million. Although the agents successfully learned to push the box rightward, the actual pushing distance is far smaller than the reward magnitude would suggest, and the anomalous physical phenomenon of the box being lifted onto robot tops persisted. From the perspectives of cognitive science and cognitive psychology, combined with the agents’ perception-decision-action loop, reward-modulated STDP (RSTDP) learning, and genetic algorithm (GA) evolution, this paper analyzes the causes of these anomalies and explores the coupling between reward structure, physical parameters, and learning strategies. Finally, improvement directions are proposed to promote stable learning in complex physical environments.

---

1. 引言

脉冲神经网络（SNN）因其生物合理性和时空信息处理能力，在机器人控制领域受到广泛关注。本研究将MSF神经元与遗传算法和RSTDP结合，构建了具备运动、大脑、通信三组神经网络的智能体，控制四轮小车在塔防风格任务中推动箱子。然而，在物理模拟过程中出现了多个与真实世界不符的异常现象，如箱子被顶起、机器人堆叠、斜坡地形下箱子飞离地面等。最终程序在平坦地面上运行，尽管箱子被向右推动，但推箱效率极低（60秒仅移动1.24米），而奖励数值却异常巨大（百万级）。这些现象不仅干扰学习，也反映了智能体策略与物理环境、奖励结构之间的复杂耦合。本文旨在从认知科学视角分析这些异常，并提出改进方案。

---

2. 游戏规划与实验设置

2.1 任务目标

游戏场景为一条30米长的水平地面（初始版本含随机斜坡，后改为平坦地面），中央放置一个质量为10kg的箱子。两队智能体（蓝队2辆，黄队3辆）分别位于箱子两侧：蓝队目标是将箱子向右推动，黄队目标是将箱子向左推动。每辆小车由独立的SNN智能体控制，通过传感器获取自身状态（位置、速度、角度）、箱子位置和速度、其他小车的位置，输出推力方向和强度。比赛持续60秒，最终以箱子位移方向与距离判断胜负。

2.2 智能体架构

每个智能体包含三个神经网络组：

• 运动组：12个神经元，输入12维传感器，输出4维（推力方向与强度）。
• 大脑组：20个神经元，输入12维传感器，输出4维（用于高级决策学习）。
• 通信组：8个神经元，输入6维（接收其他智能体的广播），输出2维（自身广播信号）。

所有突触采用多延迟结构（每个连接含3个固定延迟1ms, 2ms, 3ms），连接密度30%，初始权重随机。学习机制包括在线RSTDP（基于即时奖励更新权重）和离线遗传算法（每30秒选择前50%精英，复制并加入高斯噪声生成后代）。

2.3 奖励结构

• 位移奖励：每步箱子水平位移Δx乘以系数5（右队得正，左队得负）。
• 接触奖励：当机器人与箱子距离<0.8米时，给予±0.2奖励（鼓励推箱，抑制反向）。
• 存活奖励：机器人高度y>0.1时每步+0.002（鼓励保持直立）。
• 出界惩罚：机器人超出边界（x<0或x>30）扣10分。

2.4 物理环境

• 地面：平坦地面（最终版本），无斜坡。
• 小车：四轮驱动，底盘质量2kg，轮子半径0.2m，最大力矩80Nm，PID速度控制器（Kp=12, Ki=0, Kd=1）。
• 箱子：初始位置x=15m，y=0.6m，质量10kg，弹性系数0.1。

2.5 程序演化中的物理参数调整

阶段	修改内容	目的
初始	随机斜坡	增加地形复杂度
阶段1	取消斜坡，改为平地	消除地形导致的机器人摔倒
阶段2	降低底盘高度（0.4m→0.1m）	减少箱子被抬起的空间
阶段3	增加箱子质量（5kg→10kg）	增加惯性，降低被顶起概率
阶段4	降低弹性系数（1.0→0.1）	减少弹跳，使碰撞更“黏”
阶段5	限制推力（12N→8N）	减小冲击力
阶段6	添加垂直限位（y≤1.0）	强制箱子贴近地面

---

3. 实验结果与神经网络活动

3.1 基本统计

• 模拟时长：60.00 秒
• 总智能体数：5
• 箱子最终位置：16.24 米
• 右队累计奖励：2,391,172.91
• 左队累计奖励：-3,317,774.83

箱子向右移动1.24米，表明智能体确实学会了将箱子推向目标方向，但移动速度极慢（平均约0.02 m/s）。巨额奖励主要源于每步位移奖励的高系数（5）和大量步数（3600步），累积效应放大。实际上，如果箱子匀速移动1.24米，位移奖励仅约6.2分，但实际奖励高达239万，说明大量奖励来自接触奖励和存活奖励的反复累加，而不是推箱本身。

3.2 神经网络发放率

神经组	平均发放率 (Hz)
brain	3.9
motor	2.1

大脑组发放率（3.9 Hz）高于运动组（2.1 Hz），符合功能分化预期：大脑组负责决策，需要更频繁的信息整合；运动组执行具体动作，发放率较低。但整体发放率偏低，可能与神经元阈值较高或输入强度不足有关，也可能表明网络未充分利用脉冲编码的时序信息。

3.3 脉冲栅格图

运动组脉冲分布相对稀疏，神经元活动集中在少数时间片段；大脑组脉冲较为分散，无明显规律性。这表明运动组倾向于在特定时刻（如推箱瞬间）激活，而大脑组持续参与决策，但缺乏稳定模式，可能反映了学习尚未收敛。

3.4 权重分布

从 weights_hist_best.png 可见，权重分布近似正态，中心集中在0附近，标准差约0.5。这表明网络权重未出现极端值，学习相对稳定，但可能缺乏强关联的突触连接。

---

4. 认知科学与认知心理学分析

4.1 奖励结构对感知-决策-动作循环的误导

智能体的感知-决策-动作循环是典型的“目标导向行为”。传感器信息（自身位置、箱子位置、队友/对手位置）被输入神经网络，经过前向传播后输出推力。这一过程类似于人类在运动控制中的“前馈控制”与“反馈调整”的整合。

然而，当前的奖励结构存在严重缺陷：位移奖励系数（5）远小于接触奖励（0.2）和存活奖励（0.002）的累积效应。例如，每步存活奖励虽小，但3600步累积达7.2分，而1.24米位移奖励仅6.2分，因此存活奖励对总奖励的贡献甚至超过位移奖励。这导致智能体“认为”保持直立比推箱更重要，从而倾向于在箱子附近徘徊而不主动推进。

从认知心理学角度看，这类似于“注意偏差”或“奖励欺骗”——当任务目标被多重奖励混淆时，智能体会优先追求容易获得的即时奖励（如存活、接触），而忽视核心目标（推箱）。人类在复杂任务中同样会出现这种“短视”行为，例如在考试中只关注解题速度而忽略正确率。

4.2 多智能体对抗中的“囚徒困境”

两队机器人分别向相反方向推箱，形成了对抗性博弈。当两方推力不平衡或机器人相互挤压时，箱子可能被“夹”起。这种现象可类比于社会心理学中的“囚徒困境”：每个个体追求自身利益最大化（推箱得分），却导致集体结果恶化（箱子被抬升、双方得分降低）。

从认知科学角度，这反映了智能体缺乏“心智理论”（Theory of Mind）——它们无法推断其他智能体的意图，也无法协调合作。尽管通信组理论上允许信息交换，但实际训练中通信信号始终为0，表明智能体未能学会使用通信来协调推箱方向。这类似于人类在缺乏有效沟通时的协作失败，例如两人抬重物时若方向不一致，物体会被顶起。

4.3 神经分化与功能特化的不足

本研究设计的三个神经网络组（运动、大脑、通信）分别对应了人类认知中的运动皮层、前额叶决策区、语言区。理想情况下，运动组应学习精确的推力控制，大脑组应学习何时推、何时退，通信组应学会协调。然而，当前结果中，大脑组发放率仅略高于运动组，且通信组完全未激活，说明功能分化不充分。

在认知神经科学中，这种“功能未分化”常见于学习初期或训练不足时。例如，儿童在学习新技能时，大脑皮层激活范围广泛，随着练习，激活区域逐渐特化。本实验中，智能体可能仍处于“探索阶段”，未形成稳定、特异化的神经回路。

4.4 学习迁移与物理参数耦合

多次调整物理参数（底盘高度、质量、弹性、推力）本质上是改变了环境约束，但未改变奖励结构。智能体在物理参数变化后，原有策略可能不再适用，但遗传算法保留的精英个体仍包含“抬升”行为的权重，导致问题持续存在。

从认知心理学看，这类似于“学习迁移”中的负迁移——在一种环境中学到的策略在另一环境中反而有害。人类在学习新技能时也常遇到此类问题，例如从平地骑车过渡到山地骑行时需要重新调整平衡策略。

4.5 巨额奖励的认知解读

右队累计奖励高达239万，而箱子仅移动1.24米，这似乎自相矛盾。但分解奖励来源后可发现：

• 位移奖励：1.24 m × 5 = 6.2 分
• 接触奖励：假设每个时间步接触奖励平均0.2，则3600步×0.2=720分，但接触奖励仅当距离<0.8时触发，实际可能更少。
• 存活奖励：3600步×0.002=7.2分
• 总奖励239万远超以上总和，表明存在奖励放大现象——很可能是在遗传算法中，累计奖励被反复累加而未重置，导致数值爆炸。

从认知科学看，这类似于“激励扭曲”——当学习信号被错误放大时，智能体会“误以为”自己取得了巨大进步，而实际上进展甚微。这会导致过早收敛于局部最优，阻碍进一步探索。

---

5. 结论

本实验展示了基于SNN的智能体在推箱任务中出现的复杂物理异常与奖励误导问题。尽管智能体成功学会了将箱子向右推动，但推箱效率极低，奖励结构严重扭曲了学习目标。认知科学视角揭示了智能体在“感知-决策-动作”循环中的局限性、多智能体对抗性合作导致的不稳定、以及功能分化不足等问题。未来工作应聚焦于奖励函数的精细设计、物理参数的合理匹配、多智能体协调机制，并借鉴认知神经科学原理，使智能体学会符合物理直觉和协作规范的推箱策略。

---

c++程序

1 .demo演示和生成数据（建议在linux下）

#include <box2d/box2d.h>
#include <iostream>
#include <vector>
#include <cmath>
#include <random>
#include <fstream>
#include <algorithm>
#include <numeric>
#include <functional>
#include <memory>
#include <thread>
#include <chrono>
#include <cstdlib>

// ==================== 全局随机数 ====================
std::mt19937 rng(42);
std::uniform_real_distribution<float> dist01(0.0f, 1.0f);
std::normal_distribution<float> dist_norm(0.0f, 0.1f);

// ==================== 参数 ====================
const float DT = 1.0f / 60.0f;
const float SIMULATION_TIME = 60.0f;          // 演示时长（秒）
const float BOX_WIDTH = 0.6f;
const float BOX_HEIGHT = 0.6f;
const float BOX_MASS = 10.0f;                 // 增加箱子质量
const float CAR_MASS = 2.0f;
const float WHEEL_RADIUS = 0.2f;
const float CHASSIS_HALF_LENGTH = 0.8f;
const float CHASSIS_HALF_WIDTH = 0.5f;
const float CHASSIS_HALF_HEIGHT = 0.1f;       // 降低底盘高度
const int TOTAL_AGENTS = 5;                   // 2右 + 3左

// PID 控制器
struct PID {
    float Kp, Ki, Kd;
    float integral, prev_error;
    PID(float p, float i, float d) : Kp(p), Ki(i), Kd(d), integral(0), prev_error(0) {}
    float update(float error, float dt) {
        integral += error * dt;
        float derivative = (error - prev_error) / dt;
        prev_error = error;
        return Kp * error + Ki * integral + Kd * derivative;
    }
};

// ==================== MSF 神经元 ====================
struct DelayWeight { float delay; float weight; };

class Synapse {
public:
    std::vector<DelayWeight> delays;
    float trace_pre, trace_post;
    Synapse() : trace_pre(0), trace_post(0) {}
    void add_delay(float delay, float weight) { delays.push_back({delay, weight}); }
    void apply_psp(float pre_time, float cur_time, float& target) const {
        for (const auto& dw : delays) {
            float dt = cur_time - (pre_time + dw.delay);
            if (dt >= 0) target += dw.weight * std::exp(-dt / 10.0f);
        }
    }
    void update_traces(bool pre_spike, bool post_spike, float dt) {
        if (pre_spike) trace_pre += 1.0f;
        if (post_spike) trace_post += 1.0f;
        trace_pre *= (1 - dt / 20.0f);
        trace_post *= (1 - dt / 20.0f);
    }
    void update_rstdp(float reward, float dt, float lr) {
        float delta = reward * trace_pre * trace_post * dt * lr;
        for (auto& dw : delays) dw.weight += delta;
    }
};

class MSFNeuron {
public:
    float potential, threshold, refractory, last_spike;
    std::vector<float> spike_times;
    MSFNeuron(float thresh = 1.0f, float ref = 5.0f) : potential(0), threshold(thresh), refractory(ref), last_spike(-ref) {}
    void update(float dt, float input, float cur_time) {
        if (cur_time - last_spike < refractory) { potential = 0; return; }
        potential *= (1 - dt / 20.0f);
        potential += input * dt;
        if (potential >= threshold) {
            potential = 0;
            last_spike = cur_time;
            spike_times.push_back(cur_time);
            if (spike_times.size() > 100) spike_times.erase(spike_times.begin());
        }
    }
    bool just_spiked(float cur_time, float dt) const {
        return std::abs(last_spike - cur_time) < dt * 0.5f;
    }
};

class NeuralGroup {
public:
    std::vector<MSFNeuron> neurons;
    std::vector<std::vector<Synapse>> synapses;
    std::vector<int> input_idx, output_idx;

    NeuralGroup(int n) {
        neurons.resize(n);
        synapses.resize(n);
        for (int i = 0; i < n; ++i) synapses[i].resize(n);
    }

    void add_conn(int src, int dst, float delay, float weight) {
        synapses[src][dst].add_delay(delay, weight);
    }

    void forward(float dt, float cur_time, const std::vector<float>& inputs) {
        std::vector<float> curr(neurons.size(), 0);
        for (size_t i = 0; i < input_idx.size(); ++i) {
            if (input_idx[i] >= (int)curr.size()) {
                std::cerr << "Error: input_idx out of range\n";
                return;
            }
            curr[input_idx[i]] += inputs[i];
        }
        for (size_t src = 0; src < neurons.size(); ++src) {
            for (size_t dst = 0; dst < neurons.size(); ++dst) {
                if (src == dst) continue;
                for (float t : neurons[src].spike_times) {
                    synapses[src][dst].apply_psp(t, cur_time, curr[dst]);
                }
            }
        }
        for (size_t i = 0; i < neurons.size(); ++i) {
            neurons[i].update(dt, curr[i], cur_time);
        }
    }

    void update_traces(float dt, float cur_time) {
        for (size_t src = 0; src < neurons.size(); ++src) {
            bool pre = neurons[src].just_spiked(cur_time, dt);
            for (size_t dst = 0; dst < neurons.size(); ++dst) {
                if (src == dst) continue;
                bool post = neurons[dst].just_spiked(cur_time, dt);
                synapses[src][dst].update_traces(pre, post, dt);
            }
        }
    }

    void update_rstdp(float reward, float dt, float lr) {
        for (size_t src = 0; src < neurons.size(); ++src)
            for (size_t dst = 0; dst < neurons.size(); ++dst)
                if (src != dst) synapses[src][dst].update_rstdp(reward, dt, lr);
    }

    std::vector<float> get_outputs(float cur_time) const {
        std::vector<float> out;
        for (int idx : output_idx) {
            if (idx >= (int)neurons.size()) {
                std::cerr << "Error: output_idx out of range\n";
                continue;
            }
            int cnt = 0;
            for (float t : neurons[idx].spike_times) if (cur_time - t < 0.02f) cnt++;
            out.push_back(cnt / 0.02f);
        }
        return out;
    }

    std::vector<float> get_all_weights() const {
        std::vector<float> w;
        for (const auto& row : synapses)
            for (const auto& syn : row)
                for (const auto& dw : syn.delays)
                    w.push_back(dw.weight);
        return w;
    }

    void set_all_weights(const std::vector<float>& w, size_t& idx) {
        for (auto& row : synapses)
            for (auto& syn : row)
                for (auto& dw : syn.delays)
                    if (idx < w.size()) dw.weight = w[idx++];
    }
};

// ==================== 四轮小车 ====================
class FourWheelCar {
public:
    b2World* world;
    b2Body* chassis;
    std::vector<b2Body*> wheels;
    std::vector<b2RevoluteJoint*> wheel_joints;
    PID pid_velocity;
    float push_force;
    float target_x;

    FourWheelCar(b2World* w, float x, float y, int team) : world(w), 
        pid_velocity(12.0f, 0.0f, 1.0f), push_force(0.0f), target_x(x) {
        // 底盘（扁平）
        b2BodyDef def;
        def.type = b2_dynamicBody;
        def.position.Set(x, y);
        chassis = world->CreateBody(&def);
        b2PolygonShape chassis_shape;
        chassis_shape.SetAsBox(CHASSIS_HALF_LENGTH, CHASSIS_HALF_HEIGHT);
        b2FixtureDef fix;
        fix.shape = &chassis_shape;
        fix.density = CAR_MASS;
        fix.friction = 0.5f;
        fix.restitution = 0.1f;           // 降低弹性
        chassis->CreateFixture(&fix);

        // 四个轮子
        float wheel_x_offsets[] = { -CHASSIS_HALF_LENGTH, CHASSIS_HALF_LENGTH, -CHASSIS_HALF_LENGTH, CHASSIS_HALF_LENGTH };
        float wheel_y_offsets[] = { -CHASSIS_HALF_WIDTH, -CHASSIS_HALF_WIDTH, CHASSIS_HALF_WIDTH, CHASSIS_HALF_WIDTH };
        for (int i = 0; i < 4; ++i) {
            def.position.Set(x + wheel_x_offsets[i], y + wheel_y_offsets[i]);
            b2Body* wheel = world->CreateBody(&def);
            b2CircleShape wheel_shape;
            wheel_shape.m_radius = WHEEL_RADIUS;
            fix.shape = &wheel_shape;
            fix.density = 0.5f;
            fix.friction = 1.0f;
            fix.restitution = 0.1f;
            wheel->CreateFixture(&fix);
            wheels.push_back(wheel);

            b2RevoluteJointDef jointDef;
            jointDef.enableMotor = true;
            jointDef.maxMotorTorque = 80.0f;
            jointDef.bodyA = chassis;
            jointDef.bodyB = wheel;
            jointDef.localAnchorA.Set(wheel_x_offsets[i], wheel_y_offsets[i]);
            jointDef.localAnchorB.Set(0, 0);
            wheel_joints.push_back((b2RevoluteJoint*)world->CreateJoint(&jointDef));
        }
    }

    void setTargetX(float x) { target_x = x; }
    void setPushForce(float f) { push_force = f; }
    float getPushForce() const { return push_force; }
    b2Vec2 getCOM() { return chassis->GetPosition(); }

    void updatePID(float dt) {
        float vel = chassis->GetLinearVelocity().x;
        float pos_x = chassis->GetPosition().x;

        float error = target_x - pos_x;
        float desired_vel = error * 1.5f + push_force;
        desired_vel = std::max(-3.0f, std::min(3.0f, desired_vel));

        float velocity_error = desired_vel - vel;
        float torque = pid_velocity.update(velocity_error, dt);
        torque = std::max(-50.0f, std::min(50.0f, torque));

        for (auto joint : wheel_joints) {
            joint->SetMotorSpeed(torque);
        }
    }
};

// ==================== SNN 智能体 ====================
class Agent {
public:
    NeuralGroup motor_group, brain_group, comm_group;
    FourWheelCar* car;
    float time;
    std::vector<float> broadcast;
    float reward;
    int team;

    Agent(FourWheelCar* c, int team_id, int id) : motor_group(12), brain_group(20), comm_group(8),
        car(c), time(0), reward(0), team(team_id) {
        motor_group.input_idx = {0,1,2,3,4,5,6,7,8,9,10,11};
        motor_group.output_idx = {8,9,10,11};
        brain_group.input_idx = {0,1,2,3,4,5,6,7,8,9,10,11};
        brain_group.output_idx = {16,17,18,19};
        comm_group.input_idx = {0,1,2,3,4,5};
        comm_group.output_idx = {6,7};

        std::mt19937 local_rng(42 + id);
        std::uniform_real_distribution<float> weight_dist(-1.0f, 1.0f);
        auto init = [&](NeuralGroup& g, int n) {
            for (int i=0;i<n;i++) for (int j=0;j<n;j++) if(i!=j) {
                for (int d=0;d<3;d++) {
                    float delay = (d+1)*1.0f;
                    g.add_conn(i,j, delay, weight_dist(local_rng));
                }
            }
        };
        init(motor_group, 12);
        init(brain_group, 20);
        init(comm_group, 8);
    }

    void update(float dt, const std::vector<float>& sensors, float rwd, const std::vector<float>& received_comm) {
        time += dt;
        reward += rwd;
        motor_group.forward(dt, time, sensors);
        auto motor_out = motor_group.get_outputs(time);
        if (motor_out.size() >= 4) {
            float push_dir = motor_out[0] * 2.0f - 1.0f;
            float push_strength = motor_out[1];
            float push_force = push_dir * push_strength * 8.0f;   // 推力上限从 12 降至 8
            push_force = std::max(-8.0f, std::min(8.0f, push_force));
            car->setPushForce(push_force);
        } else {
            car->setPushForce(0.0f);
        }
        brain_group.forward(dt, time, sensors);
        comm_group.forward(dt, time, received_comm);
        broadcast = comm_group.get_outputs(time);
        motor_group.update_traces(dt, time);
        brain_group.update_traces(dt, time);
        comm_group.update_traces(dt, time);
        motor_group.update_rstdp(rwd, dt, 0.01f);
        brain_group.update_rstdp(rwd, dt, 0.01f);
        comm_group.update_rstdp(rwd, dt, 0.01f);
    }

    std::vector<float> get_all_weights() {
        auto w1 = motor_group.get_all_weights();
        auto w2 = brain_group.get_all_weights();
        auto w3 = comm_group.get_all_weights();
        w1.insert(w1.end(), w2.begin(), w2.end());
        w1.insert(w1.end(), w3.begin(), w3.end());
        return w1;
    }

    void set_all_weights(const std::vector<float>& w) {
        size_t idx = 0;
        motor_group.set_all_weights(w, idx);
        brain_group.set_all_weights(w, idx);
        comm_group.set_all_weights(w, idx);
    }
};

// ==================== 平坦地面 ====================
void createFlatGround(b2World* world, float x_start, float x_end, float y_level) {
    b2BodyDef groundDef;
    groundDef.type = b2_staticBody;
    groundDef.position.Set((x_start + x_end) / 2, y_level);
    b2Body* ground = world->CreateBody(&groundDef);
    b2PolygonShape groundShape;
    groundShape.SetAsBox((x_end - x_start) / 2, 0.1f);
    b2FixtureDef fix;
    fix.shape = &groundShape;
    fix.restitution = 0.1f;   // 地面也降低弹性
    fix.friction = 0.8f;
    ground->CreateFixture(&fix);
}

// ==================== 主程序（演示模式） ====================
int main() {
    b2Vec2 gravity(0, -9.8f);
    b2World world(gravity);

    // 平坦地面
    createFlatGround(&world, 0, 30, 0.0f);

    // 导出地面数据
    std::ofstream groundFile("ground.csv");
    groundFile << "x,y\n";
    groundFile << "0," << 0.0f << "\n";
    groundFile << "30," << 0.0f << "\n";
    groundFile.close();

    // 箱子
    b2BodyDef boxDef;
    boxDef.type = b2_dynamicBody;
    boxDef.position.Set(15.0f, 0.6f);
    b2Body* box = world.CreateBody(&boxDef);
    b2PolygonShape boxShape;
    boxShape.SetAsBox(BOX_WIDTH/2, BOX_HEIGHT/2);
    b2FixtureDef boxFix;
    boxFix.shape = &boxShape;
    boxFix.density = BOX_MASS;
    boxFix.friction = 0.5f;
    boxFix.restitution = 0.1f;   // 降低箱子弹性
    box->CreateFixture(&boxFix);

    // 创建小车（右队2个，左队3个）
    std::vector<std::unique_ptr<FourWheelCar>> cars;
    std::vector<std::unique_ptr<Agent>> agents;
    for (int i = 0; i < 2; ++i) {
        float x = 13.5f + i * 1.0f;
        cars.push_back(std::make_unique<FourWheelCar>(&world, x, 0.4f, 0));
        agents.push_back(std::make_unique<Agent>(cars.back().get(), 0, i));
    }
    for (int i = 0; i < 3; ++i) {
        float x = 16.5f - i * 1.0f;
        cars.push_back(std::make_unique<FourWheelCar>(&world, x, 0.4f, 1));
        agents.push_back(std::make_unique<Agent>(cars.back().get(), 1, 2 + i));
    }

    // 加载最佳权重（若有）
    std::vector<float> bestWeights;
    float bestFitness = -1e9;
    std::ifstream in("best_weights.txt");
    if (in) {
        in >> bestFitness;
        float v;
        while (in >> v) bestWeights.push_back(v);
        if (bestWeights.size() == agents[0]->get_all_weights().size()) {
            for (auto& a : agents) a->set_all_weights(bestWeights);
            std::cout << "Loaded best weights with fitness " << bestFitness << "\n";
        } else {
            std::cerr << "Weight size mismatch, using random initialization\n";
        }
    } else {
        std::cout << "No best weights found, using random initialization\n";
    }

    // 数据文件
    std::ofstream behaviorFile("behavior.csv");
    std::ofstream spikeFile("spikes.csv");
    std::ofstream weightFile("weights.csv");
    std::ofstream commFile("comm.csv");
    std::ofstream boxFile("box.csv");
    behaviorFile << "time,agent,team,x,y,angle,reward\n";
    spikeFile << "time,agent,group,neuron\n";
    weightFile << "generation,agent,group,synapse,weight\n";
    commFile << "time,sender,receiver,signal\n";
    boxFile << "time,x,y\n";

    float totalTime = 0;
    float lastBoxX = 15.0f;
    float lastPrintTime = 0.0f;
    const float PRINT_INTERVAL = 1.0f;

    while (totalTime < SIMULATION_TIME) {
        world.Step(DT, 8, 3);
        // 箱子离地限制
		b2Vec2 boxPos = box->GetPosition();
		if (boxPos.y > 1.0f) {
			box->SetTransform(b2Vec2(boxPos.x, 1.0f), box->GetAngle());
			box->SetLinearVelocity(b2Vec2(box->GetLinearVelocity().x, 0));
		}

        float target_offset = 1.2f;

        for (size_t i = 0; i < cars.size(); ++i) {
            float target_x;
            if (agents[i]->team == 0) target_x = boxPos.x + target_offset;
            else target_x = boxPos.x - target_offset;
            target_x = std::max(0.0f, std::min(30.0f, target_x));
            cars[i]->setTargetX(target_x);
        }

        b2Vec2 boxVel = box->GetLinearVelocity();
        float deltaX = boxPos.x - lastBoxX;
        lastBoxX = boxPos.x;

        std::vector<float> instant_rewards(agents.size(), 0.0f);
        for (size_t i = 0; i < agents.size(); ++i) {
            float rwd = 0;
            if (agents[i]->team == 0) rwd = deltaX * 5.0f;
            else rwd = -deltaX * 5.0f;

            b2Vec2 carPos = cars[i]->getCOM();
            float dist = std::hypot(carPos.x - boxPos.x, carPos.y - boxPos.y);
            if (carPos.y > 0.1f) rwd += 0.002f;
            if (dist < 0.8f) {
                rwd += (agents[i]->team == 0) ? 0.2f : -0.2f;
            }
            if (carPos.x < 0 || carPos.x > 30) rwd -= 10.0f;
            instant_rewards[i] = rwd;
            agents[i]->reward += rwd;
        }

        for (size_t i = 0; i < cars.size(); ++i) {
            cars[i]->updatePID(DT);
        }

        std::vector<b2Vec2> carPositions(agents.size());
        for (size_t i = 0; i < agents.size(); ++i) {
            carPositions[i] = cars[i]->getCOM();
        }

        std::vector<std::vector<float>> allBroadcasts(agents.size());
        for (size_t i = 0; i < agents.size(); ++i) {
            allBroadcasts[i] = agents[i]->broadcast;
        }

        for (size_t i = 0; i < agents.size(); ++i) {
            b2Vec2 selfPos = carPositions[i];
            float selfAngle = cars[i]->chassis->GetAngle();
            b2Vec2 selfVel = cars[i]->chassis->GetLinearVelocity();

            std::vector<float> sensors = {
                selfPos.x, selfPos.y, selfAngle, selfVel.x, selfVel.y,
                boxPos.x, boxPos.y, boxVel.x
            };
            for (size_t j = 0; j < agents.size(); ++j) {
                if (j != i) {
                    sensors.push_back(carPositions[j].x);
                }
            }
            while (sensors.size() < 12) sensors.push_back(0.0f);

            std::vector<float> received_comm(6, 0);
            for (size_t j = 0; j < agents.size(); ++j) {
                if (i != j) {
                    for (size_t k = 0; k < allBroadcasts[j].size() && k < received_comm.size(); ++k)
                        received_comm[k] += allBroadcasts[j][k];
                }
            }

            agents[i]->update(DT, sensors, instant_rewards[i], received_comm);
        }

        // 记录数据
        for (size_t i = 0; i < agents.size(); ++i) {
            b2Vec2 pos = cars[i]->getCOM();
            behaviorFile << totalTime << "," << i << "," << agents[i]->team << ","
                         << pos.x << "," << pos.y << "," << cars[i]->chassis->GetAngle() << ","
                         << agents[i]->reward << "\n";

            for (size_t n = 0; n < agents[i]->motor_group.neurons.size(); ++n) {
                if (agents[i]->motor_group.neurons[n].just_spiked(agents[i]->time, DT))
                    spikeFile << totalTime << "," << i << ",motor," << n << "\n";
            }
            for (size_t n = 0; n < agents[i]->brain_group.neurons.size(); ++n) {
                if (agents[i]->brain_group.neurons[n].just_spiked(agents[i]->time, DT))
                    spikeFile << totalTime << "," << i << ",brain," << n << "\n";
            }
            for (size_t n = 0; n < agents[i]->comm_group.neurons.size(); ++n) {
                if (agents[i]->comm_group.neurons[n].just_spiked(agents[i]->time, DT))
                    spikeFile << totalTime << "," << i << ",comm," << n << "\n";
            }

            for (size_t k = 0; k < agents[i]->broadcast.size(); ++k) {
                commFile << totalTime << "," << i << ",broadcast," << k << ","
                         << agents[i]->broadcast[k] << "\n";
            }
        }

        boxFile << totalTime << "," << boxPos.x << "," << boxPos.y << "\n";

        if (totalTime - lastPrintTime >= PRINT_INTERVAL) {
            lastPrintTime = totalTime;
            float score_right = 0, score_left = 0;
            int alive_right = 0, alive_left = 0;
            float avg_x_right = 0, avg_x_left = 0;
            int count_right = 0, count_left = 0;
            for (size_t i = 0; i < agents.size(); ++i) {
                b2Vec2 pos = cars[i]->getCOM();
                if (agents[i]->team == 0) {
                    score_right += agents[i]->reward;
                    if (pos.y > 0.1f) alive_right++;
                    avg_x_right += pos.x;
                    count_right++;
                } else {
                    score_left += agents[i]->reward;
                    if (pos.y > 0.1f) alive_left++;
                    avg_x_left += pos.x;
                    count_left++;
                }
            }
            if (count_right > 0) avg_x_right /= count_right;
            if (count_left > 0) avg_x_left /= count_left;
            std::cout << "Time: " << totalTime << " s | Box X: " << boxPos.x
                      << " | Right: " << score_right << " (" << alive_right << "/2 alive, avg X=" << avg_x_right << ")"
                      << " | Left: " << score_left << " (" << alive_left << "/3 alive, avg X=" << avg_x_left << ")\n";
        }

        totalTime += DT;
        std::this_thread::sleep_for(std::chrono::milliseconds(16));
    }

    behaviorFile.close();
    spikeFile.close();
    weightFile.close();
    commFile.close();
    boxFile.close();
    return 0;
}

2.python演示

import pandas as pd
import matplotlib.pyplot as plt
import matplotlib.animation as animation
from matplotlib.widgets import Slider
import numpy as np
# 设置中文字体
plt.rcParams['font.sans-serif'] = ['SimHei', 'DejaVu Sans']  # 用来正常显示中文标签
plt.rcParams['axes.unicode_minus'] = False  # 用来正常显示负号
# ==================== 数据读取 ====================
behavior = pd.read_csv('behavior.csv')
box_data = pd.read_csv('box.csv')
ground = pd.read_csv('ground.csv')

# 排序
behavior = behavior.sort_values(['time', 'agent'])
time = behavior['time'].unique()
agents = behavior['agent'].unique()
teams = {agent: behavior[behavior['agent'] == agent]['team'].iloc[0] for agent in agents}
colors = {0: 'blue', 1: 'orange'}

# 地面
if not ground.empty:
    ground = ground.sort_values('x')
    ground_x = ground['x'].values
    ground_y = ground['y'].values
else:
    ground_x = np.array([])
    ground_y = np.array([])

# 箱子数据
box_time = box_data['time'].values
box_x = box_data['x'].values
box_y = box_data['y'].values

# ==================== 绘图设置 ====================
y_min, y_max = -2, 10
x_min, x_max = 0, 30
fig, ax = plt.subplots(figsize=(12, 6))
plt.subplots_adjust(bottom=0.2)  # 为滑块留出空间
ax.set_xlim(x_min, x_max)
ax.set_ylim(y_min, y_max)
ax.set_xlabel('X (m)')
ax.set_ylabel('Y (m)')
ax.grid(True, alpha=0.3)

# 地面
if len(ground_x) > 0:
    ax.plot(ground_x, ground_y, 'k-', linewidth=2.5, label='地面')
    ax.legend(loc='upper left')

# 箱子
box_width, box_height = 0.8, 0.6
box_rect = plt.Rectangle((0, 0), box_width, box_height, fc='saddlebrown',
                         ec='black', label='箱子', zorder=3, alpha=0.8)
ax.add_patch(box_rect)

# 机器人
scatters = {}
texts = {}
for agent in agents:
    scat = ax.scatter([], [], s=80, c=colors[teams[agent]],
                      edgecolors='black', zorder=5, linewidth=1.2)
    scatters[agent] = scat
    text = ax.text(0, 0, '', fontsize=10, ha='center', va='bottom',
                   zorder=7, weight='bold')
    texts[agent] = text

# 移动检测文本
movement_text = ax.text(x_min + 0.02*(x_max-x_min), y_max - 0.03*(y_max-y_min),
                        '', fontsize=10,
                        bbox=dict(boxstyle="round,pad=0.5", facecolor="lightblue", alpha=0.7),
                        verticalalignment='top', zorder=10)
speed_text = ax.text(x_min + 0.2*(x_max-x_min), y_max - 0.03*(y_max-y_min),
                     '', fontsize=10,
                     bbox=dict(boxstyle="round,pad=0.5", facecolor="lightyellow", alpha=0.7),
                     verticalalignment='top', zorder=10)

# ==================== 动画控制 ====================
current_frame = 0
play_speed = 30  # 默认帧间隔（毫秒）
ani = None

def update(frame):
    t = time[frame]
    df_t = behavior[behavior['time'] == t]

    # 箱子位置
    closest_idx = np.argmin(np.abs(box_time - t))
    box_rect.set_xy((box_x[closest_idx] - box_width/2, box_y[closest_idx] - box_height/2))

    # 移动检测（可选）
    x_movements, y_movements = [], []
    if frame > 0:
        prev_t = time[frame-1]
        df_prev = behavior[behavior['time'] == prev_t]
        for agent in agents:
            curr = df_t[df_t['agent'] == agent]
            prev = df_prev[df_prev['agent'] == agent]
            if not curr.empty and not prev.empty:
                cx, cy = curr['x'].iloc[0], curr['y'].iloc[0]
                px, py = prev['x'].iloc[0], prev['y'].iloc[0]
                if abs(cx - px) > 0.01: x_movements.append(agent)
                if abs(cy - py) > 0.01: y_movements.append(agent)

    # 机器人
    for agent in agents:
        data = df_t[df_t['agent'] == agent]
        if not data.empty:
            x = data['x'].iloc[0]
            y = data['y'].iloc[0]
            scatters[agent].set_offsets([[x, y]])
            texts[agent].set_position((x, y + (y_max-y_min)*0.03))
            texts[agent].set_text(f'{agent}')
        else:
            scatters[agent].set_offsets([[-1000, -1000]])
            texts[agent].set_position((-1000, -1000))
            texts[agent].set_text('')

    ax.set_title(f'时间: {t:.2f} s', fontsize=14, pad=20)
    movement_text.set_text(f"X轴移动: {', '.join(map(str, x_movements)) if x_movements else '无'}")
    speed_text.set_text(f"Y轴移动: {', '.join(map(str, y_movements)) if y_movements else '无'}")

    # 更新进度滑块（如果存在）
    if 'slider_frame' in globals():
        slider_frame.set_val(frame / (len(time)-1))

    return list(scatters.values()) + list(texts.values()) + [box_rect, movement_text, speed_text]

def init():
    return update(0)

def on_speed_change(val):
    """速度滑块回调：调整帧间隔"""
    global play_speed
    play_speed = int(val)
    ani.event_source.interval = play_speed

def on_frame_change(val):
    """进度滑块回调：跳转到指定帧"""
    frame = int(val * (len(time)-1))
    update(frame)
    # 强制重绘
    fig.canvas.draw_idle()

# 创建动画（初始帧间隔 30ms）
ani = animation.FuncAnimation(fig, update, frames=len(time), init_func=init,
                              interval=30, blit=True, repeat=True)

# 添加滑块
ax_speed = plt.axes([0.2, 0.05, 0.6, 0.03])   # [left, bottom, width, height]
speed_slider = Slider(ax_speed, '速度', 5, 100, valinit=30, valstep=1)
speed_slider.on_changed(on_speed_change)

ax_frame = plt.axes([0.2, 0.01, 0.6, 0.03])
frame_slider = Slider(ax_frame, '进度', 0, 1, valinit=0, valstep=0.01)
frame_slider.on_changed(on_frame_change)

plt.show()

3. 参数

727.334
-0.769891
-0.00627744
0.218133
0.239879
0.300832
0.331595
-0.518821
-0.655735
-0.345722
1.6767
1.7729
0.204105
1.42368
0.181609
1.17383
1.27304
0.106801
1.23843
0.467497
-0.498779
-0.2101
0.385258
1.69462
0.113094
0.600295
1.02768
0.205223
2.00055
1.82437
1.20263
0.533562
1.39095
0.901482
0.161754
-0.367807
-0.662236
-0.846675
-0.168262
0.686449
0.306011
0.697878
0.241143
0.94293
-0.19246
-0.229246
0.747532
0.908976
0.11013
-0.108483
-0.142178
0.339449
-0.720615
-0.835
-0.539521
0.794197
-0.605704
-0.403993
0.596984
-0.47539
-0.335341
-0.989741
-0.157808
0.086405
-0.110357
-0.0488073
0.649535
1.3068
0.359359
1.99046
-0.206612
0.817325
0.748606
0.820306
0.926849
0.0505114
1.61194
0.229748
0.258614
0.39844
1.91172
1.9427
0.324471
0.841461
0.963433
0.730034
0.217996
0.344355
0.563927
1.29247
1.87885
0.587826
1.90128
1.83021
0.305717
0.449857
1.6451
0.845422
1.87035
2.02376
0.0648677
0.471454
-0.384087
-0.109877
0.209672
0.121326
0.552034
-0.177489
-0.254526
0.453976
0.130451
-0.201606
-0.89856
0.34029
-0.126921
0.409431
0.55173
0.21912
-0.999277
0.0800689
0.648319
-0.587836
0.623236
-0.601677
0.794848
0.591478
0.0338172
-0.419334
0.59264
0.311926
-0.194054
-0.400766
0.839008
0.343579
1.4487
0.862535
-0.894858
-0.379461
0.921432
0.508507
1.51158
1.19792
-0.133555
-0.509313
0.985743
1.55234
1.21682
1.49709
1.84359
1.4305
1.90002
-0.794514
-0.382993
0.887285
1.56594
1.06202
1.93379
1.8562
0.322912
0.454009
1.88739
1.2452
0.588738
1.98762
0.905651
2.0542
1.13518
0.139841
1.95905
-0.0373933
0.790398
-0.417155
0.627894
0.164052
1.84945
0.139249
0.279006
-0.989833
1.90301
1.27936
0.582454
1.79139
1.64156
1.81823
-0.827466
0.908639
0.641413
1.57256
1.93968
0.462266
1.08425
0.964809
1.4297
1.8666
1.15237
0.604895
0.658285
0.982137
1.45567
1.28265
0.476516
0.255479
-0.613678
-0.0197807
-0.640921
0.106964
0.670603
0.640811
0.826128
0.504079
-0.860239
1.47097
1.59972
1.92308
0.823026
1.60482
0.0777836
-0.569543
-0.435212
0.809942
1.58967
1.15405
0.184978
1.74088
1.77621
1.89606
1.28234
1.70844
0.968892
0.839423
1.38763
1.2474
0.217214
-0.134604
-0.702021
0.388294
-0.0846733
-0.030627
-0.146211
-0.743479
0.277689
0.288741
-0.908524
-0.590588
-0.516196
0.489914
0.692693
-0.788766
-0.0378766
-0.985381
0.280911
-0.331022
-0.0255085
0.938511
0.151399
0.401951
-0.162057
-0.172629
0.156747
-0.758496
0.955395
-0.879273
0.553786
0.452423
0.798497
1.90142
1.95036
0.122574
-0.412073
0.168282
-0.739179
1.49211
0.496079
0.619295
0.0398104
0.0455953
-0.465483
1.68505
1.71349
1.248
0.941721
0.849952
1.25976
0.204636
0.933922
1.02333
0.446077
-0.0636122
-0.327698
0.455857
1.29528
0.233059
1.64204
1.10922
0.14396
0.756544
1.85666
0.521836
0.307284
0.280768
0.653449
0.675861
0.559777
-0.289775
1.96003
1.69028
1.91273
-0.948075
0.580072
-0.995521
1.97126
1.90569
0.665415
1.44332
1.64325
2.08508
1.49734
1.01009
1.1358
-0.681725
-0.545834
0.904566
1.60643
0.424928
0.856204
0.833382
1.88502
1.96393
1.52502
0.258736
1.72641
0.998999
1.33418
1.86566
-0.423103
-0.36119
0.0064826
1.96139
0.518811
2.03082
-0.223548
-0.0279689
-0.175909
0.698887
0.106551
2.00318
1.43308
2.09224
1.80453
0.940946
1.60169
0.140011
0.159964
-0.105878
-0.4541
1.51276
0.315645
1.34456
0.415707
1.43611
0.977764
2.03864
0.432751
1.41874
1.99758
0.274405
0.862675
0.179462
-0.357038
-0.576838
1.97222
1.62677
0.67221
-0.299388
-0.921139
0.00487292
1.04398
1.46388
1.56249
0.438766
1.54276
1.29763
1.21445
0.371097
1.76055
0.653157
-0.915045
-0.334597
1.44917
1.18623
0.676182
1.75687
2.05665
2.03359
0.814874
0.749873
1.18932
0.0987689
0.487953
1.849
0.352505
2.0383
1.08777
0.838154
-0.0369722
0.700971
-0.969059
-0.0126145
-0.5355
1.031
1.56389
1.21167
1.40155
0.227263
0.127952
1.84753
0.993842
1.28036
1.24266
2.07743
0.657145
1.47844
1.16713
0.802757
1.79429
0.4401
0.170937
0.180097
1.7114
0.741338
1.526
0.753162
1.11102
0.975303
0.358087
1.33448
1.48453
0.230931
0.859824
-0.407109
-0.00341833
-0.593154
0.254627
1.3337
0.483723
1.1465
1.2032
1.3558
1.3055
0.348969
1.66012
0.0997761
0.105953
0.133184
0.0690527
0.83545
0.302048
1.66169
0.635403
0.984782
0.0483162
0.0509865
0.0918756
0.0884526
-0.512904
0.032838
1.71187
1.69731
1.73002
0.703561
0.311018
1.52222
1.84141
1.56744
0.960129
1.11218
1.36121
0.320645
1.55839
1.47457
1.62626
-0.0654314
1.1326
0.0291991
0.69769
1.08126
1.22316
0.401259
1.53877
0.425656
0.358245
-0.0278283
0.00277176
1.70048
0.237953
1.02173
0.373687
-0.952796
0.490047
1.03946
0.861645
1.30017
1.23151
1.28754
1.19928
0.696181
1.15863
0.654255
1.63257
0.457723
0.144723
1.94853
0.58943
1.63861
1.40985
1.89026
0.249988
0.815815
0.295817
-0.185143
-0.511549
-0.126471
0.079631
1.30874
1.07111
0.710184
1.00507
0.0598032
0.97577
0.369655
1.48636
1.15343
1.41826
1.13508
1.80261
2.00465
0.835806
1.12425
0.910081
1.39579
0.20429
0.599868
0.276918
0.843286
0.24457
0.171405
1.70022
0.882806
1.40571
0.392732
0.94954
0.177479
0.906982
0.586629
-0.311233
-0.719039
1.60681
1.80046
0.422291
0.341765
1.826
1.5985
1.76013
1.64924
0.652883
0.0655617
0.569261
1.28273
0.211251
0.619627
-0.599736
0.855926
-0.851053
0.166141
-0.978801
0.567003
-0.624294
0.00935805
0.767965
0.157392
0.320649
-0.528693
-0.744542
-0.143694
-0.304035
0.300198
-0.75282
-0.435118
-0.17179
-0.122993
0.637152
0.889805
-0.135477
-0.786219
0.368849
-0.198749
0.888688
0.589656
-0.976337
-0.7944
-0.354142
0.107179
-0.0819366
-0.298201
-0.1607
0.0570482
-0.945838
-0.343199
0.756335
-0.846782
0.207044
0.789448
-0.882961
0.96715
-0.721021
-0.14801
0.0905447
0.410441
-0.348877
-0.157174
0.0909477
0.526599
0.893039
0.595116
-0.498245
-0.384042
-0.387266
-0.913153
-0.525313
0.990713
0.895315
0.407565
0.255488
0.444771
-0.695989
0.302492
0.886521
-0.869017
-0.485966
-0.895948
0.0998328
-0.0202715
-0.858595
0.611886
-0.782378
0.958236
0.932526
0.164109
0.821339
-0.813221
-0.0640042
-0.30015
-0.852406
0.885821
-0.29649
0.311527
-0.663514
-0.133106
-0.596172
0.922851
0.392152
0.50965
-0.520468
-0.671726
-0.455471
-0.712842
-0.137526
0.311027
0.765959
0.959481
-0.808458
0.621028
0.516873
0.432021
0.739053
0.539482
0.00152659
-0.674168
-0.659674
-0.022504
0.812099
0.222315
1.58955
0.472478
0.626004
0.0325528
0.525014
0.447716
1.94634
1.83684
0.365869
-0.917477
-0.0953711
-0.423635
0.0678207
-0.812081
0.956651
1.37811
1.40438
0.520465
1.83791
1.04193
0.42609
0.68811
1.39848
1.85959
0.639627
1.53969
1.91494
0.491389
1.01028
1.33557
1.59238
0.359255
1.18191
0.639521
1.97208
0.771758
0.379529
0.838519
0.369214
1.93146
1.56307
0.16191
-0.782347
0.617974
-0.782228
1.28277
0.343977
1.67001
1.71261
0.517891
1.02694
-0.0460441
0.815004
0.0771807
1.63944
0.217976
1.24682
0.352077
1.2413
0.361158
0.806022
1.27029
1.14179
1.33457
1.2158
0.0581209
-0.475397
0.127694
-0.989929
0.860282
-0.874981
0.146601
1.1762
0.876864
0.168553
0.288827
0.644401
1.2632
1.11452
0.789614
1.93828
0.6172
1.90823
0.247834
0.87283
0.619036
1.5612
0.349308
0.773108
0.701909
0.145828
0.3613
1.35276
1.29292
1.78501
1.61275
1.8145
1.50997
1.97121
-0.16106
-0.00316745
-0.512671
1.48534
1.11528
1.65554
1.72802
1.82513
1.41444
0.254234
1.25397
1.05094
0.741225
0.869027
0.344464
1.53234
0.391431
1.25828
1.88951
1.0396
0.231067
0.105988
1.72616
1.22838
0.556208
-0.737956
0.1256
-0.158554
0.825662
0.172254
1.88376
1.90126
1.75675
1.31367
0.310305
-0.0105957
0.709059
1.40599
1.0294
1.5925
0.841457
1.57227
0.252754
2.02186
1.74738
0.560787
0.629952
1.82899
0.663567
0.894028
1.5826
0.651641
0.670294
1.05949
1.66949
1.57132
0.276383
0.440919
0.544249
0.0448837
0.739714
2.00747
0.547122
0.308748
1.15225
0.747334
0.505919
1.18352
1.17859
1.88415
1.47169
1.72963
1.87924
1.83971
0.732729
1.65627
1.78757
0.586264
2.03264
1.04032
1.1973
-0.561115
-0.261175
-0.43806
0.919728
-0.22229
0.612239
0.475557
1.22502
1.05874
1.37383
0.54994
1.6869
0.403692
0.63948
1.95592
0.335863
0.403918
1.1357
1.62515
0.990829
1.73728
0.476669
1.80425
0.899515
1.06763
1.04468
1.95517
1.00138
1.65047
1.13205
1.00954
1.07283
2.00935
0.657021
0.627455
-0.893109
1.18459
1.94206
1.82039
0.224455
0.920943
0.0653652
1.18423
1.83816
0.741182
1.77886
0.849654
1.97532
1.76943
2.08301
0.579455
1.09424
1.41994
1.02133
1.47442
0.581692
1.33593
-0.762226
0.18208
0.616358
0.664815
0.373927
0.645411
0.500376
1.53231
1.87661
1.9543
1.05376
1.59747
0.441848
1.97929
1.05298
0.344548
0.328987
1.39549
1.29752
1.48378
1.73245
0.27745
0.570123
1.76156
1.15966
2.01834
0.387925
1.45325
1.60421
1.50952
0.655688
1.9839
1.99996
-0.90349
0.140814
0.304491
0.824173
1.68089
1.1819
1.53868
0.0210724
0.0377133
1.51347
0.588115
1.14092
1.12327
1.87778
2.02987
1.9902
1.50156
1.71505
0.0607808
0.0827464
1.13854
2.0215
1.30302
1.54339
0.0914432
0.452609
0.519225
-0.234338
-0.132699
-0.848784
0.312199
1.0489
0.865109
0.127562
1.40684
1.16795
1.89422
1.61625
1.0997
1.05723
1.60189
1.11005
0.691795
0.687665
0.90688
0.114538
1.52145
1.03593
1.9661
1.51841
0.136573
0.134459
1.58282
1.71456
1.81433
0.58344
0.893421
-0.109913
-0.950377
-0.0230137
1.75058
0.413894
0.772706
0.0686055
0.427863
0.881787
0.719269
0.376144
0.952967
2.03647
2.02615
0.226492
0.963269
1.48167
0.318709
1.56246
0.728946
0.990742
1.53726
0.777466
1.81092
-0.911293
0.728314
0.185695
-0.22064
0.648505
-0.785164
0.524281
2.03193
0.360369
0.427073
1.22003
1.33467
1.12227
1.17286
0.31003
2.02411
1.24737
0.966993
1.36859
0.388671
0.486946
0.430425
1.42337
0.337714
1.51724
0.0502517
0.449052
0.784034
0.498104
1.14518
1.56536
1.81452
0.239938
0.0227274
-0.163712
-0.070592
1.6446
0.29634
0.669376
1.79724
0.235341
1.38836
0.681156
0.516518
0.855027
1.92849
0.482479
1.28144
0.0604525
1.66928
0.770506
0.538781
0.156639
0.193439
0.353276
0.160379
1.59746
-0.232085
-0.121493
-0.568113
-0.495081
-0.109589
-0.440825
0.260083
1.33812
0.101681
1.82888
0.0139864
1.97565
1.53754
0.992298
0.789845
1.19709
0.262774
0.10907
0.54725
2.01265
0.753137
2.02588
2.0366
0.521179
0.404658
1.39384
1.06642
1.45747
0.71139
1.37698
0.491304
1.1447
1.78734
-0.0202562
-0.748396
-0.133237
0.232923
1.15239
1.00237
1.35662
1.50887
1.49797
1.68917
1.18391
0.367802
0.398976
1.65638
1.71489
0.377304
0.656542
1.27499
0.442787
0.301287
0.451012
1.97389
1.00354
1.30329
0.409993
-0.564209
0.529999
0.916084
0.41621
-0.00778389
0.110746
0.781255
0.732606
1.89357
1.36838
1.53
1.10536
1.06393
1.87978
0.545584
1.34825
0.0863382
1.87146
1.69358
0.572953
0.317597
1.56137
1.22469
0.284335
2.00623
1.39563
1.83476
1.24599
0.689917
1.32961
0.684883
1.67422
0.823932
-0.0102186
-0.625099
1.35028
1.11574
0.545182
0.596285
0.8969
1.15433
1.65301
0.356061
1.25422
1.481
0.184783
0.226543
0.261686
0.0801153
0.993224
0.0856006
1.62992
1.07215
1.12743
0.539197
1.24828
-0.0339255
0.259041
-0.96046
-0.678865
-0.553002
0.635999
0.9148
1.05868
1.26351
0.153604
0.809839
1.0519
1.97659
0.267873
0.961968
1.53317
1.30437
0.890452
0.499657
0.706298
1.84623
0.344809
1.70868
1.80281
0.350849
0.170469
1.40125
1.29455
0.0679078
1.40811
0.548229
1.93006
0.706284
-0.299075
-0.750058
0.298163
0.662672
0.481162
0.321315
0.564671
1.32923
1.5855
0.115917
0.971131
-0.0655817
0.065129
0.27794
1.16632
-0.804919
0.154392
0.321951
0.0836657
-0.254069
-0.549335
-0.92344
-0.870619
-0.574126
-0.370408
-0.908458
0.0766386
0.798242
0.25138
0.0968903
-0.573228
-0.596357
0.423124
0.377042
-0.90821
-0.0292428
-0.30554
0.700743
0.0545143
-0.854477
0.656281
-0.157966
0.528254
-0.32916
0.25205
0.970732
-0.206251
-0.346043
0.894524
0.908335
-0.756547
-0.187735
0.873982
0.555217
0.2754
0.544274
0.424726
-0.350616
-0.134188
-0.0404937
0.0210941
0.437938
0.245448
0.885294
-0.239304
0.817195
0.335056
-0.880129
-0.806813
-0.357135
0.0674989
0.449974
0.225745
0.140603
0.896311
0.380009
1.82755
0.889607
0.277906
1.31615
1.82646
-0.980288
-0.322115
0.0204859
-0.216556
0.122808
-0.283884
1.28456
1.77765
0.421361
0.526911
0.393586
0.889436
0.15634
1.44298
1.60716
0.0840034
1.60673
0.946883
0.8372
1.26446
1.97501
0.150302
0.676561
1.73213
1.83766
0.625448
2.01597
1.86416
1.26977
0.73996
1.83883
1.34353
1.16917
1.5438
0.250802
1.45488
0.490157
-0.703791
0.967456
0.731984
1.52014
1.79258
1.35555
0.470872
0.327171
2.00463
1.88584
1.63329
1.52641
1.79979
1.64193
0.806663
0.0255901
1.42232
0.124426
0.32177
1.95019
-0.548877
-0.27648
0.792127
0.40374
0.31755
-0.511297
1.49586
1.12698
1.99166
0.538536
1.67056
1.5948
0.833821
0.663798
0.844723
1.35598
1.29482
0.988853
1.46821
0.38556
1.226
0.271122
0.407679
1.97971
0.733117
1.41049
1.79506
0.772446
0.344993
0.00455536
1.43703
0.35494
1.6506
1.30975
1.51149
0.742624
-0.639523
-0.716308
-0.365557
0.478688
1.83841
0.441201
0.494943
0.550369
1.67615
1.57611
1.50427
-0.0170191
1.81295
0.495355
0.726254
1.54783
0.848076
1.39298
-0.0401167
-0.0721271
0.192745
0.192546
0.157035
0.551791
-0.938201
-0.829066
-0.927203
1.63152
0.316473
1.26987
0.717605
1.34401
0.170309
1.65118
0.966047
0.7819
0.00775023
1.13782
0.613712
1.01828
0.61704
0.0926752
0.537644
0.634246
0.494858
-0.075404
0.858947
1.07471
0.163878
0.566359
1.02842
0.155565
1.19005
1.42013
1.11322
1.30227
0.618638
0.864373
-0.152446
0.765752
1.75843
1.79113
1.19382
0.992041
0.637702
0.525797
0.643245
1.85125
0.165791
0.438201
1.77559
0.0786672
1.70957
1.15007
0.99414
0.915926
1.52083
0.811654
0.601683
-0.0230072
0.828523
0.384156
0.0018661
-0.535255
1.93318
1.72104
1.69589
1.14176
0.235356
1.29652
1.05948
1.6951
1.67673
0.736355
2.05389
0.602201
0.736081
1.52957
0.528791
1.56239
1.33167
0.278975
0.104757
1.02872
0.202083
0.383754
1.54125
1.58904
1.73357
1.22814
1.67938
1.71937
1.12627
1.82195
0.811055
-0.878988
0.927674
0.676535
1.25602
0.364863
1.31911
1.30639
1.77967
0.878498
1.4436
1.45415
0.760484
0.910915
0.198338
-0.951026
-0.349716
-0.377953
-0.214498
0.255286
-0.546825
0.507088
-0.978108
0.0210623
-0.970859
-0.438333
0.65959
0.738567
-0.327607
0.658148
0.249901
-0.601376
0.116835
-0.196195
-0.161323
-0.851132
-0.419909
0.677301
-0.149313
-0.935941
-0.0400677
0.277522
0.738756
0.293298
-0.500823
-0.915643
-0.824389
0.641022
0.965808
0.51422
-0.0819048
-0.372174
0.479238
0.274004
-0.949413
0.842404
0.319983
-0.516893
-0.856169
0.202038
-0.234866
0.64958
-0.000547528
0.553408
-0.260325
-0.83907
-0.369985
-0.403639
0.828301
0.737733
-0.45429
-0.367185
-0.565571
-0.158024
0.958559
-0.380598
-0.166046
-0.968431
0.24592
-0.913007
0.817769
0.237149
0.418045
-0.684888
-0.159365
-0.718051
0.751039
-0.545733
0.581219
0.298794
0.765143
0.780924
0.556238
0.698326
0.624609
-0.791741
0.87369
-0.309534
-0.546
-0.924688
-0.010388
-0.529386
-0.108999
-0.463644
-0.493083
-0.723705
0.0804873
0.893344
-0.151644
-0.0805063
0.546924
0.820994
-0.380243
0.986611
0.387728
0.0455494
0.403393
-0.813568
-0.910356
-0.659965
-0.664073
0.112385
-0.426786
-0.479518
0.458193
-0.374823
0.815671
0.0146577
0.706054
0.77304
-0.22368
0.127759
-0.570935
-0.415365
0.71549
0.993062
0.251655
-0.790748
-0.376278
-0.124334
-0.812816
-0.624303
0.0618379
0.910478
0.494543
-0.360876
0.919654
-0.824947
0.173581
0.356242
-0.184586
-0.535997
-0.819881
-0.116555
0.180231
0.339065
-0.394999
-0.948541
-0.0547552
-0.910815
0.146771
-0.396792
0.7673
0.491078
0.960346
0.115909
0.321815
-0.839527
0.602611
-0.80304
-0.969122
-0.141118
-0.355759
0.0821934
0.169247
-0.224977
-0.708755
0.112944
0.616046
0.16994

参考文献

[1] Gerstner, W., & Kistler, W. M. (2002). Spiking Neuron Models: Single Neurons, Populations, Plasticity. Cambridge University Press.
[2] Sutton, R. S., & Barto, A. G. (2018). Reinforcement Learning: An Introduction. MIT Press.
[3] Friston, K. (2010). The free-energy principle: a unified brain theory? Nature Reviews Neuroscience, 11(2), 127–138.
[4] Catania, A. C. (2013). Learning (5th ed.). Sloan Publishing.
[5] Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux.
[6] Frith, C. D., & Frith, U. (2006). The neural basis of mentalizing. Neuron, 50(4), 531–534.
[7] Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285.
[8] Diamond, A. (2013). Executive functions. Annual Review of Psychology, 64, 135–168.