LAB 03 · RELEASE 2026-02-17

实验 3:Raft

Lab 3: Raft

按 3A–3D 四个里程碑实现 Raft:选主、日志复制、持久化和日志压缩。页面完整保留接口、阶段边界、调试方法与公开测试命令,但不提供可提交实现。

DUE 3A 02-27;3B 03-06;3C 03-13;3D 04-0308 MILESTONES03 SOURCES

边界、依赖与验收

分四阶段实现选主、日志复制、持久化与快照支持的 Raft 复制状态机。

RELEASE

开始 Raft 四阶段

LEC 06

选主与心跳 → 3A

3A DUE

Leader election

LEC 07

日志复制、持久化与快照

3B DUE

Log replication

LEC 10

Lab Q&A 与时间线调试

3C DUE

Persistence

3D DUE

Log compaction

实验总览:构建 Raft 模块本页可离线完成

Handout 完整本土化

这是容错 KV 系列的第一个实验。本实验实现复制状态机协议 Raft;下一实验在其上构建 KV 服务,之后再把服务分片到多个复制状态机以提高性能。

复制服务把完整状态副本保存在多个服务器。服务器崩溃或网络损坏时,只要多数派存活且能通信,服务继续;没有多数派时不推进,恢复通信后从原状态继续。难点是故障会让日志不同。

Raft 把客户端请求组织为带 index 的日志,保证副本看到相同的 committed log,并按序 apply 到各自服务状态。恢复节点由 Raft 补齐。你的实现是供更大服务调用的 Go 对象,通过 RPC 维护无限命令序列;每个 entry 提交后通过 applyCh 交给服务。

严格依据 extended Raft paper,尤其 Figure 2。实现选举、日志复制、持久化、重启恢复和快照;不实现 Section 6 成员变更。本实验分 3A–3D 四次按期提交。

代码、测试、接口与通信限制本页可离线完成

Handout 完整本土化

若已完成 Lab 1,仓库已存在;否则按 Lab 1 克隆。骨架是 src/raft1/raft.go,测试是 src/raft1/raft_test.go。先更新并运行:

cd ~/6.5840
git pull
cd src
make raft1

初始测试会在 TestInitialElection3A 报告没有 leader,并生成可视化时间线。

只在 raft1/raft.go 添加实现。必须支持:

rf := Make(peers, me, persister, applyCh)
rf.Start(command interface{}) (index, term, isleader)
rf.GetState() (term, isLeader)
type ApplyMsg

peers 含所有 Raft peer 网络标识,me 是本机下标。Start 立即返回,不等待复制完成。每个新提交 entry 都通过 Make 传入的 applyCh 发送 ApplyMsg。

RPC 使用 src/labrpc;测试器可延迟、重排、丢弃。可临时改 labrpc 调试,但评分用原版。Raft 实例只能通过 RPC 通信,禁止共享 Go 变量或文件。后续实验依赖本实现,请留足时间写清晰代码。评分不带 -race,但你必须用 -race 自测。

Part 3A:leader election 与 heartbeat本页可离线完成

Handout 完整本土化

实现 leader election 和空日志 AppendEntries heartbeat。目标:正常时选出并维持一个 leader;旧 leader 故障或收发包丢失时,只要多数派可通信,5 秒内选出新 leader。

按 Figure 2 添加选举状态;补全 RequestVoteArgs/Reply,让 Make 启动后台 goroutine,在一段时间没听到 peer 后发起选举;实现 RequestVote handler。定义 AppendEntries RPC(暂可不含全部参数),leader 周期发送,follower 实现 handler。别忘记 GetState()

测试限制 heartbeat 最多每秒 10 次。论文 150–300 ms election timeout 依赖更频繁 heartbeat,不适合本测试;你的 timeout 要大于 heartbeat 间隔,又不能大到 5 秒内无法重选。可用 rand 随机化。

课程建议用循环配合 time.Sleep(),参考 Make 创建的 ticker();不要用难以正确重置的 time.Timer/time.Ticker。Figure 2 的选举逻辑分散在多个区域,失败时重新逐条核对。

Go RPC 只传大写开头的导出字段,日志记录等嵌套结构也一样;不要忽略 labgob 警告。失败时测试会生成含分区、崩溃、检查事件的可视化;可调用 tester.Annotate("Server 0", "short description", "details") 加注释。

运行:

make RUN="-run 3A" raft1

必须通过 TestInitialElection3ATestReElection3ATestManyElections3A。每条 Passed 后数字依次为时间、peer 数、RPC 数、RPC 字节数和 committed entry 数,可用于检查 RPC 是否异常。Lab 3–5 全套超过 600 秒或单测超过 120 秒会评分失败。

Part 3B:日志复制、提交与 apply本页可离线完成

Handout 完整本土化

实现 leader/follower 追加日志,使全部 3B 测试通过。开始前 git pull。论文用 1-based log;课程建议 Go slice 用 index 0 的 term 0 dummy entry,使第一条 AppendEntries 的 PrevLogIndex=0 始终可索引。

先通过 TestBasicAgree3B:实现 Start(),再依 Figure 2 发送/接收带 entries 的 AppendEntries,并在每个 peer 把新 committed entry 发到 applyCh。还必须实现 Section 5.4.1 election restriction,避免缺少已提交日志的 candidate 当选。

反复检查事件的循环不能空转;用 condition variable 或每轮 time.Sleep(10*time.Millisecond)。为后续实验写清晰代码。失败时从 raft_test.go 追入测试逻辑理解条件。

运行:

make RUN="-run 3B" raft1

测试包括 TestBasicAgree3BTestRPCBytes3BTestFollowerFailure3BTestLeaderFailure3BTestFailAgree3BTestFailNoAgree3BTestConcurrentStarts3BTestRejoin3BTestBackup3BTestCount3B。典型全套约 72 秒;若超过几分钟,检查无效 sleep、RPC timeout 等待、busy loop 或过量 RPC。

Start() 只表示本地 leader 接受提议,不表示提交。commitIndex 依据多数派 matchIndex 和 current-term 规则推进;apply 必须按 index 顺序。

Part 3C:持久状态与重启本页可离线完成

Handout 完整本土化

Raft server 重启后应从原位置继续。真实系统每次持久状态变化写磁盘;本实验使用 tester1/persister.go 的 Persister。Make 收到的 Persister 初始包含上次状态,Raft 要读出,并在每次 persistent state 变化时保存。

完成 persist()readPersist(),使用 ReadRaftState()Save() 和 labgob 编解码字节。labgob 类似 gob,但会提示小写字段。当前阶段 persister.Save() 第二参数传 nil。必须在所有改变 currentTerm、votedFor、log 的路径调用 persist,并保证成功 RPC 回复前状态已保存。

3C 通常需要快速回退优化。follower reject 可返回:

XTerm  冲突 entry 的 term(若有)
XIndex 该 term 的第一条 index(若有)
XLen   follower log 长度

leader:若无 XTerm,nextIndex=XIndex;若有,跳到 leader 中该 term 最后一条之后;follower 太短则 nextIndex=XLen。论文细节模糊,需要自行补全但保持边界检查。

3C 更严格,失败可能来自 3A/3B。运行:

make RUN="-run 3C" raft1

必须通过 TestPersist13CTestPersist23CTestPersist33CTestFigure83CTestUnreliableAgree3CTestFigure8Unreliable3CTestReliableChurn3CTestUnreliableChurn3C,并保留 3A/3B 全通过。建议多次运行再提交。

Part 3D:日志压缩与 Snapshot(index)本页可离线完成

Handout 完整本土化

永久保留并重放完整日志不可行。服务会周期性持久保存状态 snapshot,然后 Raft 丢弃 snapshot 之前的 entry,减少持久数据和重启时间。若 follower 落后到 leader 已删掉所需日志,leader 发送 snapshot 再发送之后日志。依据扩展论文 Section 7 自行补全细节。

Raft 向服务提供:

Snapshot(index int, snapshot []byte)

index 是 snapshot 已包含的最高 log entry。Lab 3D 由 tester 在每个 peer 调用;Lab 4 由 KV service 调用,内容是完整 KV table。实现 trimmed log 后所有 Raft 逻辑 index 与 slice offset 都要转换。

实现 InstallSnapshot RPC。follower 收到后通过 applyCh 的 ApplyMsg 把 snapshot 交给服务;raftapi/raftapi.go 已定义字段。只允许 snapshot 推进服务,不能让乱序 RPC 把状态倒退。

崩溃恢复必须同时保存 Raft state 与对应 snapshot,使用 persister.Save() 第二参数;无 snapshot 才传 nil。重启时应用先恢复持久 snapshot,此后 applyCh 第一条必须是更高 SnapshotIndex 的 snapshot,或紧接恢复 index 的普通 command。

建议先让日志支持从 X 开始但初始 X=0,跑 3B/3C;再让 Snapshot 丢弃 index 前内容,通过首个 3D 测试。follower 追赶太慢是首测常见失败。下一步当 nextIndex 落在 leader 保留范围之前时发送 InstallSnapshot。

本实验整个 snapshot 用单个 RPC,不实现 Figure 13 offset 分块。丢弃 slice 必须去掉所有可达引用,让 Go GC 回收。Make 应 ReadSnapshot() 并在每次 Save 带上非 nil snapshot(只要已 trim)。

3D 与全套性能验收本页可离线完成

Handout 完整本土化

运行:

make RUN="-run 3D" raft1

测试为 TestSnapshotBasic3DTestSnapshotInstall3DTestSnapshotInstallUnreliable3DTestSnapshotInstallCrash3DTestSnapshotInstallUnCrash3DTestSnapshotAllCrash3DTestSnapshotInit3D。同时必须继续通过 3A/3B/3C。

官方给出的合理时间:不带 -race 的 Lab 3 全套约 6 分钟真实时间、1 分钟 CPU;带 -race 约 10 分钟真实时间、2 分钟 CPU。评分不用 race,但你应稳定通过 race。

提交各 part 前按提交说明使用对应 make lab3a 等打包,并确认此前 part 全部仍通过。

官方调试与可视化使用要点本页可离线完成

Handout 完整本土化

测试失败时先打开测试器生成的 HTML timeline,沿 term、leader、分区、崩溃和检查事件定位最早偏离。用 tester.Annotate 标出选举、commit、snapshot 等自己的关键状态变化,避免只打印海量 heartbeat。

使用 go test -race;锁保护共享状态,但不要在持锁时等待 RPC 或 applyCh。为 currentTerm、commitIndex、lastApplied、matchIndex、snapshotIndex 等写单调性断言。Kill 后所有后台 goroutine 应停止。

错误常在早期 part 潜伏,到 churn/snapshot 才暴露;每次修改后回跑已有 part。测试器与可视化是观察工具,Figure 2 与论文规则才是规范。

完整官方 Handout 与配套资料

以下是本讲对应官方材料的可搜索离线文本。中文精读负责解释;资料附录保留原始细节、例子、问答与代码,不以摘要替代原文。

网页讲义labs/lab-raft1.html618 行 · 3,979 词 · 完整收录
6.5840 Lab 3: Raft

6.5840 - Spring 2026

6.5840 Lab 3: Raft

  Collaboration policy //
  Submit lab //
  Setup Go //
  Guidance //
  Piazza

Introduction

This is the first in a series of labs in which you'll build a
fault-tolerant key/value storage system. In this
lab you'll implement Raft, a replicated state machine protocol.
In the next lab you'll build a key/value service on top of
Raft. Then you will “shard” your service over
multiple replicated state machines for higher performance.

A replicated service achieves fault
tolerance by storing complete copies of its state (i.e., data)
on multiple replica servers.
Replication allows
the service to continue operating even if some of
its servers experience failures (crashes or a broken or flaky
network). The challenge is that failures may cause the
replicas to hold differing copies of the data.

Raft organizes client requests into a sequence, called
the log, and ensures that all the replica servers see the same log.
Each replica executes client requests
in log order, applying them to its local copy of the service's state.
Since all the live replicas
see the same log contents, they all execute the same requests
in the same order, and thus continue to have identical service
state. If a server fails but later recovers, Raft takes care of
bringing its log up to date. Raft will continue to operate as
long as at least a majority of the servers are alive and can
talk to each other. If there is no such majority, Raft will
make no progress, but will pick up where it left off as soon as
a majority can communicate again.

In this lab you'll implement Raft as a Go object type
with associated methods, meant to be used as a module in a
larger service. A set of Raft instances talk to each other with
RPC to maintain replicated logs. Your Raft interface will
support an indefinite sequence of numbered commands, also
called log entries. The entries are numbered with index
numbers. The log entry with a given index will eventually
be committed. At that point, your Raft should send the log
entry to the larger service for it to execute.

You should follow the design in the
extended Raft paper,
with particular attention to Figure 2.
You'll implement most of what's in the paper, including saving
persistent state and reading it after a node fails and
then restarts. You will not implement cluster
membership changes (Section 6).

This lab is due in four parts. You must submit each part on the
corresponding due date.

Getting Started

If you have done Lab 1, you already have a copy of the lab
source code.
If not,
you can find directions for obtaining the source via git
in the Lab 1 instructions.

We supply you with skeleton code src/raft1/raft.go. We also
supply a set of tests, which you should use to drive your
implementation efforts, and which we'll use to grade your submitted
lab. The tests are in src/raft1/raft_test.go.

When we grade your submissions, we will run the tests without the -race flag.
However, you should test with -race.

To get up and running, execute the following commands.
Don't forget the git pull to get the latest software.

$ cd ~/6.5840
$ git pull
...
$ cd src
$ make raft1
go build -race -o main/raft1d main/raft1d.go
cd raft1 && go test -v -race
=== RUN   TestInitialElection3A
Test (3A): initial election (reliable network)...
Fatal: expected one leader, got none
        /Users/rtm/824-process-raft/src/raft1/test.go:151
        /Users/rtm/824-process-raft/src/raft1/raft_test.go:36
info: wrote visualization to /var/folders/x_/vk0xmxwn1sj91m89wsn5b1yh0000gr/T/porcupine-2242138501.html
--- FAIL: TestInitialElection3A (5.51s)
...
$

The code

Implement Raft by adding code to
raft1/raft.go. In that file you'll find
skeleton code, plus examples of how to send and receive
RPCs.

Your implementation must support the following interface, which
the tester and (eventually) your key/value server will use.
You'll find more details in comments in raft.go
and in raftapi/raftapi.go.

// create a new Raft server instance:
rf := Make(peers, me, persister, applyCh)

// start agreement on a new log entry:
rf.Start(command interface{}) (index, term, isleader)

// ask a Raft for its current term, and whether it thinks it is leader
rf.GetState() (term, isLeader)

// each time a new entry is committed to the log, each Raft peer
// should send an ApplyMsg to the service (or tester).
type ApplyMsg

A service calls Make(peers,me,…) to create a
Raft peer. The peers argument is an array of network identifiers
of the Raft peers (including this one), for use with RPC. The
me argument is the index of this peer in the peers
array. Start(command) asks Raft to start the processing
to append the command to the replicated log. Start()
should return immediately, without waiting for the log appends
to complete. The service expects your implementation to send an
ApplyMsg for each newly committed log entry to the
applyCh channel argument to Make().

raft.go contains example code that sends an RPC
(sendRequestVote()) and that handles an incoming RPC
(RequestVote()).
Your Raft peers should exchange RPCs using the labrpc Go
package (source in src/labrpc).
The tester can tell labrpc to delay RPCs,
re-order them, and discard them to simulate various network failures.
While you can temporarily modify labrpc,
make sure your Raft works with the original labrpc,
since that's what we'll use to test and grade your lab.
Your Raft instances must interact only through RPC; for example,
they are not allowed to communicate using shared Go variables
or files.

Subsequent labs build on this lab, so it is important to give
yourself enough time to write solid code.

Part 3A: leader election

Implement Raft leader election and heartbeats (AppendEntries RPCs with no
log entries). The goal for Part 3A is for a
single leader to be elected, for the leader to remain the leader
if there are no failures, and for a new leader to take over if the
old leader fails or if packets to/from the old leader are lost.
Run make RUN="-run 3A" raft1 in the src
directory to test your 3A code.

Follow the paper's Figure 2. At this point you care about sending
and receiving RequestVote RPCs, the Rules for Servers that relate to
elections, and the State related to leader election,

Add the Figure 2 state for leader election
to the Raft struct in raft.go.

Fill in the RequestVoteArgs and
RequestVoteReply structs. Modify
Make() to create a background goroutine that will kick off leader
 election periodically by sending out RequestVote RPCs when it hasn't
 heard from another peer for a while.
 Implement
 the RequestVote() RPC handler so that servers will vote for one
 another.

To implement heartbeats, define an
AppendEntries RPC struct (though you may not
need all the arguments yet), and have the leader send
them out periodically. Write an
AppendEntries RPC handler method.

The tester requires that the leader send heartbeat RPCs no more than
ten times per second.

The tester requires your Raft to elect a new leader within five
seconds of the failure of the old leader (if a majority of peers can
still communicate).

The paper's Section 5.2 mentions election timeouts in the range of 150
to 300 milliseconds. Such a range only makes sense if the leader
sends heartbeats considerably more often than once per 150
milliseconds (e.g., once per 10 milliseconds). Because the tester limits you tens of heartbeats per
second, you will have to use an election timeout larger
than the paper's 150 to 300 milliseconds, but not too large, because then you
may fail to elect a leader within five seconds.

You may find Go's
rand
useful.

You'll need to write code that takes actions periodically or
after delays in time. The easiest way to do this is to create
a goroutine with a loop that calls
time.Sleep();
see the ticker() goroutine that Make()
creates for this purpose.
Don't use Go's time.Timer or time.Ticker, which
are difficult to use correctly.

If your code has trouble passing the tests,
read the paper's Figure 2 again; the full logic for leader
election is spread over multiple parts of the figure.

Don't forget to implement GetState().

Go RPC sends only struct fields whose names start with capital letters.
  Sub-structures must also have capitalized field names (e.g. fields of log records
  in an array). The labgob package will warn you about this;
  don't ignore the warnings.

The most challenging part of this lab may be the debugging. Refer to
the Guidance page for debugging tips.

If you fail a test, the tester produces a file that visualizes a timeline with
events marked along it, including network partitions, crashed servers, and
checks performed. Here's an example of the
visualization. Further, you can add your own annotations by writing, for
example,
tester.Annotate("Server 0", "short description", "details").

Be sure you pass the 3A tests before submitting Part 3A, so that
you see something like this:

$ make RUN="-run 3A" raft1
go build -race -o main/raft1d main/raft1d.go
cd raft1 && go test -v -race -run 3A
=== RUN   TestInitialElection3A
Test (3A): initial election (reliable network)...
  ... Passed --  time  3.5s #peers 3 #RPCs    32 #Ops    0
--- PASS: TestInitialElection3A (3.84s)
=== RUN   TestReElection3A
Test (3A): election after network failure (reliable network)...
  ... Passed --  time  6.2s #peers 3 #RPCs    68 #Ops    0
--- PASS: TestReElection3A (6.54s)
=== RUN   TestManyElections3A
Test (3A): multiple elections (reliable network)...
  ... Passed --  time  9.8s #peers 7 #RPCs   684 #Ops    0
--- PASS: TestManyElections3A (10.68s)
PASS
ok      6.5840/raft1    22.095s
$

Each "Passed" line contains five numbers; these are the time that the
test took in seconds, the number of Raft peers, the
number of RPCs sent during the test, the total number of bytes in the
RPC messages, and the number of log entries
that Raft reports were committed. Your numbers will differ from those
shown here. You can ignore the numbers if you like, but they may help
you sanity-check the number of RPCs that your implementation sends.
For all of labs 3, 4, and 5, the grading script will fail your
solution if it takes more than 600 seconds for all of the tests,
or if any individual test takes more than 120
seconds.

When we grade your submissions, we will run the tests without
the -race
flag. However, you should make sure that your code consistently
passes the tests with the -race flag.

Part 3B: log

Implement the leader and follower code to append new log entries,
so that make RUN="-run 3B" raft1 passes all tests.

Run git pull to get the latest lab software.

The Raft paper views the log as 1-indexed, but we suggest that you implement
it as 0-indexed, starting
with a dummy entry at index=0 that has term 0. That allows the very
first AppendEntries RPC to contain 0 as PrevLogIndex, and be a valid index into
the log.

Your first goal should be to pass TestBasicAgree3B().
Start by implementing Start(), then write the code
to send and receive new log entries via AppendEntries RPCs,
following Figure 2. Send each newly committed entry
on applyCh on each peer.

You will need to implement the election
restriction (section 5.4.1 in the paper).

Your code may have loops that repeatedly check for certain events.
Don't have these loops
execute continuously without pausing, since that
will slow your implementation enough that it fails tests.
Use Go's
condition variables,
or insert a
time.Sleep(10 * time.Millisecond) in each loop iteration.

Do yourself a favor for future labs and write (or re-write) code
that's clean and clear.

If you fail a test, look at
  raft_test.go and trace the test code from there to
  understand what's being tested.

The tests for upcoming labs may fail your code if it runs too slowly.
You can check how much real time and CPU time your solution uses with
the time command. Here's typical output:

$ make RUN="-run 3B" raft1
go build -race -o main/raft1d main/raft1d.go
cd raft1 && go test -v -race -run 3B
=== RUN   TestBasicAgree3B
Test (3B): basic agreement (reliable network)...
  ... Passed --  time  1.6s #peers 3 #RPCs    18 #Ops    3
--- PASS: TestBasicAgree3B (1.96s)
=== RUN   TestRPCBytes3B
Test (3B): RPC byte count (reliable network)...
  ... Passed --  time  3.3s #peers 3 #RPCs    50 #Ops   11
--- PASS: TestRPCBytes3B (3.71s)
=== RUN   TestFollowerFailure3B
Test (3B): test progressive failure of followers (reliable network)...
  ... Passed --  time  5.4s #peers 3 #RPCs    58 #Ops    3
--- PASS: TestFollowerFailure3B (5.77s)
=== RUN   TestLeaderFailure3B
Test (3B): test failure of leaders (reliable network)...
  ... Passed --  time  6.5s #peers 3 #RPCs   110 #Ops    3
--- PASS: TestLeaderFailure3B (6.89s)
=== RUN   TestFailAgree3B
Test (3B): agreement after follower reconnects (reliable network)...
  ... Passed --  time  6.0s #peers 3 #RPCs    61 #Ops    7
--- PASS: TestFailAgree3B (6.37s)
=== RUN   TestFailNoAgree3B
Test (3B): no agreement if too many followers disconnect (reliable network)...
  ... Passed --  time  4.0s #peers 5 #RPCs   107 #Ops    2
--- PASS: TestFailNoAgree3B (4.55s)
=== RUN   TestConcurrentStarts3B
Test (3B): concurrent Start()s (reliable network)...
  ... Passed --  time  1.4s #peers 3 #RPCs    12 #Ops    0
--- PASS: TestConcurrentStarts3B (1.75s)
=== RUN   TestRejoin3B
Test (3B): rejoin of partitioned leader (reliable network)...
  ... Passed --  time  7.8s #peers 3 #RPCs   120 #Ops    4
--- PASS: TestRejoin3B (8.15s)
=== RUN   TestBackup3B
Test (3B): leader backs up quickly over incorrect follower logs (reliable network)...
  ... Passed --  time 27.7s #peers 5 #RPCs  1370 #Ops  102
--- PASS: TestBackup3B (28.27s)
=== RUN   TestCount3B
Test (3B): RPC counts aren't too high (reliable network)...
  ... Passed --  time  2.7s #peers 3 #RPCs    32 #Ops    0
--- PASS: TestCount3B (3.05s)
PASS
ok      6.5840/raft1    71.716s
$

The "ok 6.5840/raft 71.716s" means that Go measured the time taken for the 3B
tests to be 71.716 seconds of real (wall-clock) time.
If your solution uses much more than a few minutes of real time
for the 3B tests, you may run
into trouble later on. Look for time spent sleeping or waiting for RPC
timeouts, loops that run without sleeping or waiting for conditions or
channel messages, or large numbers of RPCs sent.

Part 3C: persistence

If a Raft-based server reboots it should resume service
where it left off. This requires
that Raft keep persistent state that survives a reboot. The
paper's Figure 2 mentions which state should be persistent.

A real implementation would write
Raft's persistent state to disk each time it changed, and would read the
state from
disk when restarting after a reboot. Your implementation won't use
the disk; instead, it will save and restore persistent state
from a Persister object (see tester1/persister.go).
Whoever calls Raft.Make() supplies a Persister
that initially holds Raft's most recently persisted state (if
any). Raft should initialize its state from that
Persister, and should use it to save its persistent
state each time the state changes. Use the Persister's
ReadRaftState() and Save() methods.

Complete the functions
persist()
and
readPersist() in raft.go
by adding code to save and restore persistent state. You will need to encode
(or "serialize") the state as an array of bytes in order to pass it to
the Persister. Use the labgob encoder;
see the comments in persist() and readPersist().
labgob is like Go's gob encoder but
prints error messages if
you try to encode structures with lower-case field names.
For now, pass nil as the second argument to persister.Save().
Insert calls to persist() at the points where
your implementation changes persistent state.
Once you've done this,
and if the rest of your implementation is correct,
you should pass all of the 3C tests.

You will probably need the optimization that backs up
nextIndex by more than one entry
at a time. Look at the extended Raft paper starting at
the bottom of page 7 and top of page 8 (marked by a gray line).
The paper is vague about the details; you will need to fill in the gaps.
One possibility is to have a rejection message include:

    XTerm:  term in the conflicting entry (if any)
    XIndex: index of first entry with that term (if any)
    XLen:   log length

Then the leader's logic can be something like:

  Case 1: leader doesn't have XTerm:
    nextIndex = XIndex
  Case 2: leader has XTerm:
    nextIndex = (index of leader's last entry for XTerm) + 1
  Case 3: follower's log is too short:
    nextIndex = XLen

A few other hints:

Run git pull to get the latest lab software.

The 3C tests are more demanding than those for 3A or 3B, and failures
may be caused by problems in your code for 3A or 3B.

Your code should pass all the 3C tests (as shown below), as well as
the 3A and 3B tests.

$ make RUN="-run 3C" raft1
go build -race -o main/raft1d main/raft1d.go
cd raft1 && go test -v -race -run 3C
=== RUN   TestPersist13C
Test (3C): basic persistence (reliable network)...
  ... Passed --  time  7.6s #peers 3 #RPCs    58 #Ops    6
--- PASS: TestPersist13C (7.99s)
=== RUN   TestPersist23C
Test (3C): more persistence (reliable network)...
  ... Passed --  time 21.6s #peers 5 #RPCs   287 #Ops   16
--- PASS: TestPersist23C (22.17s)
=== RUN   TestPersist33C
Test (3C): partitioned leader and one follower crash, leader restarts (reliable network)...
  ... Passed --  time  3.8s #peers 3 #RPCs    30 #Ops    4
--- PASS: TestPersist33C (4.11s)
=== RUN   TestFigure83C
Test (3C): Figure 8 (reliable network)...
  ... Passed --  time 48.5s #peers 5 #RPCs   499 #Ops    2
--- PASS: TestFigure83C (49.08s)
=== RUN   TestUnreliableAgree3C
Test (3C): unreliable agreement (unreliable network)...
  ... Passed --  time  5.1s #peers 5 #RPCs   288 #Ops  246
--- PASS: TestUnreliableAgree3C (5.68s)
=== RUN   TestFigure8Unreliable3C
Test (3C): Figure 8 (unreliable) (unreliable network)...
  ... Passed --  time 53.6s #peers 5 #RPCs  3200 #Ops    2
--- PASS: TestFigure8Unreliable3C (54.19s)
=== RUN   TestReliableChurn3C
Test (3C): churn (reliable network)...
  ... Passed --  time 18.2s #peers 5 #RPCs  1701 #Ops    1
--- PASS: TestReliableChurn3C (18.80s)
=== RUN   TestUnreliableChurn3C
Test (3C): unreliable churn (unreliable network)...
  ... Passed --  time 17.3s #peers 5 #RPCs  1253 #Ops    1
--- PASS: TestUnreliableChurn3C (17.92s)
PASS
ok      6.5840/raft1    180.983s
$

It is a good idea to run the tests multiple times before submitting.

Part 3D: log compaction

As things stand now, a rebooting server replays the
complete Raft log in order to restore its state. However, it's not
practical for a long-running service to remember the complete Raft log
forever. Instead, you'll modify Raft to cooperate with services that
persistently store a "snapshot" of their state from time to time, at
which point Raft discards log entries that precede the snapshot. The
result is a smaller amount of persistent data and faster restart.
However, it's now possible for a follower to fall so far behind that
the leader has discarded the log entries it needs to catch up; the
leader must then send a snapshot plus the log starting at the time of
the snapshot. Section 7 of the
extended Raft paper
outlines the scheme; you will have to design the details.

Your Raft must provide the following function that the service
can call with a serialized snapshot of its state:

Snapshot(index int, snapshot []byte)

In Lab 3D, the tester calls Snapshot() periodically. In Lab 4, you will
write a key/value server that calls Snapshot(); the snapshot
will contain the complete table of key/value pairs.
The service layer calls Snapshot() on every peer (not
just on the leader).

The index argument indicates the highest log entry that's
reflected in the snapshot. Raft should discard its log entries before
that point. You'll need to revise your Raft code to operate while
storing only the tail of the log.

You'll need to implement the InstallSnapshot RPC discussed in
the paper that allows a Raft leader to tell a lagging Raft peer to
replace its state with a snapshot. You will likely need to think
through how InstallSnapshot should interact with the state and rules
in Figure 2.

When a follower's Raft code receives an InstallSnapshot RPC, it can
use the applyCh to send the snapshot to the service in
an ApplyMsg. The ApplyMsg struct definition
in raftapi/raftapi.go already
contains the fields you will need (and which the tester expects). Take
care that these snapshots only advance the service's state, and don't
cause it to move backwards.

If a server crashes, it must restart from persisted data. Your Raft
should persist both Raft state and the corresponding snapshot.
Use the second argument to
persister.Save() to save the snapshot.
If there's no snapshot, pass nil as the second
argument.

When a server restarts, the application layer reads the persisted
snapshot and restores its saved application state. After a restart,
the application layer expects the first message on applyCh to either
contain a snapshot with a SnapshotIndex higher than that of the
initial restored snapshot, or an ordinary command with CommandIndex
immediately following the index of the initial restored snapshot.

Implement Snapshot() and the InstallSnapshot RPC, as well as the
changes to Raft to support these (e.g, operation with a
trimmed log).  Your solution is complete when it passes the 3D tests
(and all the previous Lab 3 tests).

 git pull to make sure you have the latest software.

 A good place to start is to modify your code to so that it is
able to store just the part of the log
starting at some index X. Initially you can set X to zero and
run the 3B/3C tests.
Then make Snapshot(index) discard the log before index,
and set X equal to index. If all goes well you should
now pass the first 3D test.

A common reason for failing the first 3D test is that followers take too long to
catch up to the leader.

Next: have the leader send an InstallSnapshot RPC if it doesn't
have the log entries required to bring a follower up to date.

Send the entire snapshot in a single InstallSnapshot RPC.
Don't implement Figure 13's offset mechanism for
splitting up the snapshot.

 Raft must discard old log entries in a way that allows the Go garbage collector to free and re-use the
memory; this requires that there be no reachable references (pointers)
to the discarded log entries.

When a Raft peer is re-started, the persister passed to Make()
will contain a snapshot of application state as well as Raft's saved
state. Raft must include a non-nil snapshot with every call
to persister.Save() (if the log has been trimmed), which
means that it's a good idea for
Make() to call persister.ReadSnapshot() and save the
result.

A reasonable amount of time to consume for the full set of
Lab 3 tests (3A+3B+3C+3D) without -race is 6 minutes of real time and one
minute of CPU time. When running with -race, it is about 10 minutes of real
time and two minutes of CPU time.

Your code should pass all the 3D tests (as shown below), as well as the 3A, 3B, and 3C tests.

$ make RUN="-run 3D" raft1
go build -race -o main/raft1d main/raft1d.go
cd raft1 && go test -v -race -run 3D
=== RUN   TestSnapshotBasic3D
Test (3D): snapshots basic (reliable network)...
  ... Passed --  time  8.4s #peers 3 #RPCs   279 #Ops   31
--- PASS: TestSnapshotBasic3D (8.74s)
=== RUN   TestSnapshotInstall3D
Test (3D): install snapshots (disconnect) (reliable network)...
  ... Passed --  time 59.6s #peers 3 #RPCs   919 #Ops   91
--- PASS: TestSnapshotInstall3D (59.99s)
=== RUN   TestSnapshotInstallUnreliable3D
Test (3D): install snapshots (disconnect) (unreliable network)...
  ... Passed --  time 82.1s #peers 3 #RPCs  1083 #Ops   91
--- PASS: TestSnapshotInstallUnreliable3D (82.49s)
=== RUN   TestSnapshotInstallCrash3D
Test (3D): install snapshots (crash) (reliable network)...
  ... Passed --  time 53.6s #peers 3 #RPCs   685 #Ops   91
--- PASS: TestSnapshotInstallCrash3D (53.99s)
=== RUN   TestSnapshotInstallUnCrash3D
Test (3D): install snapshots (crash) (unreliable network)...
  ... Passed --  time 66.2s #peers 3 #RPCs   717 #Ops   91
--- PASS: TestSnapshotInstallUnCrash3D (66.60s)
=== RUN   TestSnapshotAllCrash3D
Test (3D): crash and restart all servers (unreliable network)...
  ... Passed --  time 20.4s #peers 3 #RPCs   244 #Ops   45
--- PASS: TestSnapshotAllCrash3D (20.79s)
=== RUN   TestSnapshotInit3D
Test (3D): snapshot initialization after crash (unreliable network)...
  ... Passed --  time  7.4s #peers 3 #RPCs    79 #Ops   14
--- PASS: TestSnapshotInit3D (7.77s)
PASS
ok      6.5840/raft1    301.406s
$
网页讲义labs/guidance.html103 行 · 597 词 · 完整收录
Lab guidance


Lab guidance

Hardness of assignments


Each lab task is tagged to indicate
roughly how long we expect the task to take:

    Easy: A few hours.

    Moderate: ~ 6 hours (per week).

    Hard: More than 6 hours (per week). If
    you start late, your solution is unlikely to pass all tests.



Most of the labs require only a modest amount of code
(perhaps a
few hundred lines per lab part), but can be conceptually difficult
and may require a good deal of thought and debugging.
Some of the tests are difficult to pass.

Don't start a lab the night before it is due; it's more
efficient to do the labs in several sessions spread over multiple
days. Tracking down bugs in distributed systems is difficult,
because of concurrency, crashes, and an unreliable network.

Tips


Do the Online Go tutorial and
  consult
  Effective Go.
    See Editors to
    set up your editor for Go.

The lab Makefiles are set up to use
Go's race detector.
Fix any races it reports.

Advice on locking in labs.

Advice on structuring your Raft lab.

This Diagram of Raft interactions may
help you understand code flow
between different parts of the system.

Learn about Go's Printfformat strings:
Go format strings.

 To learn more about git, look at the
Pro Git book or the
git user's manual.

Debugging

Efficient debugging takes experience. It
helps to be systematic: form a hypothesis about a possible cause of the
problem; collect evidence that might be relevant; think about the
information you've gathered; repeat as needed. For extended debugging
sessions it helps to keep notes, both to accumulate evidence and to
remind yourself why you've discarded specific earlier hypotheses.

The most effective debugging technique is often to add print
statements to your code, run the test that is failing and collect the
print output in a file, and then look through the output file to
identify the point at which things start to go wrong. You may need to
iterate, adding more print statements as you learn more about what is
going wrong.

Concurrency among different peers and among the threads in a single
peer can cause actions to be interleaved in unexpected ways. For
example, it's quite possible for a Raft peer to be elected leader
while the previous leader still thinks it is the leader, or for a
leader to send an RPC but receive the reply after it has lost
leadership. Adding print statements may help you spot such situations.

Feel free to examine the test code (mr/mt_test.go,
raft1/raft_test.go, &c) to understand what the tests are
exploring. You can add print statements to the tests to help you
understand what they are doing and why they are failing, but be sure
you code passes with the original test code before submitting.

The Raft paper's Figure 2 must be followed fairly exactly. It is easy
to miss a condition that Figure 2 says must be checked, or a state
change that it says must be made. If you have a bug, re-check that all
of your code adheres closely to Figure 2.

As you're writing code (i.e., before you have a bug), it may be worth
adding explicit checks for conditions that the code assumes to be
true, perhaps using Go's
panic. Such checks may
help detect situations where later code unwittingly violates the
assumptions.

The TAs are happy to help you think about your code during office
hours, but you're likely to get the most mileage out of limited office
hour time if you've already dug as deep as you can into the situation.
网页讲义labs/vis.html16 行 · 13 词 · 完整收录
Porcupine





        Clients

        Time



        Valid LP

        Invalid LP
        [ jump to first error ]