首页 > IT职业 > IT职场 > 用雪花算法就不会产生重复的ID?
确实,它简单、高效、趋势递增,但你知道吗?
雪花算法的隐蔽的坑不少。
今天这篇文章跟大家一起聊聊雪花算法的 5大坑,希望对你会有所帮助。
先简单回顾一下雪花算法的结构。
标准的雪花算法ID由64位组成:
// 典型的雪花算法结构public class SnowflakeId { // 64位ID结构
// 1位符号位(始终为0) +
// 41位时间戳(毫秒级) +
// 10位机器ID +
// 12位序列号
private long timestampBits = 41; // 时间戳占41位
private long workerIdBits = 10; // 机器ID占10位
private long sequenceBits = 12; // 序列号占12位
// 最大支持值
private long maxWorkerId = -1L ^ (-1L << workerIdBits); // 1023
private long maxSequence = -1L ^ (-1L << sequenceBits); // 4095
// 偏移量
private long timestampShift = sequenceBits + workerIdBits; // 22
private long workerIdShift = sequenceBits; // 12}
看起来很美,对吧?
但美丽的背后,是五个需要警惕的深坑:

接下来,我们逐一深入分析这五个坑。
有一天,我们线上订单系统突然出现大量重复ID。
排查后发现,有一台服务器的时间被NTP服务自动校准, 时钟回拨了2秒钟。
// 有问题的雪花算法实现public synchronized long nextId() { long currentTimestamp = timeGen();
// 问题代码:如果发现时钟回拨,直接抛异常
if (currentTimestamp < lastTimestamp) { throw new RuntimeException("时钟回拨异常");
}
// ... 生成ID的逻辑}
结果就是:时钟回拨的那台服务器完全不可用,所有请求都失败。
时钟为什么会回拨?
在分布式系统中, 你无法保证所有服务器时钟完全一致,这是物理限制。
public class SnowflakeIdWorker { private long lastTimestamp = -1L; private long sequence = 0L;
public synchronized long nextId() { long timestamp = timeGen();
// 处理时钟回拨
if (timestamp < lastTimestamp) { long offset = lastTimestamp - timestamp;
// 如果回拨时间较小(比如5毫秒内),等待
if (offset <= 5) { try {
wait(offset << 1); // 等待两倍时间
timestamp = timeGen(); if (timestamp < lastTimestamp) { throw new RuntimeException("时钟回拨过大");
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt(); throw new RuntimeException("等待时钟同步被中断");
}
} else { // 回拨过大,抛出异常
throw new RuntimeException("时钟回拨过大: " + offset + "ms");
}
}
// 正常生成ID的逻辑
if (lastTimestamp == timestamp) {
sequence = (sequence + 1) & maxSequence; if (sequence == 0) {
timestamp = tilNextMillis(lastTimestamp);
}
} else {
sequence = 0L;
}
lastTimestamp = timestamp;
return ((timestamp - twepoch) << timestampShift) |
(workerId << workerIdShift) |
sequence;
}
// 等待下一个毫秒
private long tilNextMillis(long lastTimestamp) { long timestamp = timeGen(); while (timestamp <= lastTimestamp) {
timestamp = timeGen();
} return timestamp;
}
}
// 将部分workerId位用作回拨计数器public class SnowflakeWithBackward { // 调整位分配:39位时间戳 + 13位机器ID + 3位回拨计数 + 9位序列号
private static final long BACKWARD_BITS = 3L; // 支持最多7次回拨
private long backwardCounter = 0L; // 回拨计数器
public synchronized long nextId() { long timestamp = timeGen();
if (timestamp < lastTimestamp) { // 时钟回拨,增加回拨计数器
backwardCounter = (backwardCounter + 1) & ((1 << BACKWARD_BITS) - 1);
if (backwardCounter == 0) { // 回拨计数器溢出,抛出异常
throw new RuntimeException("时钟回拨次数过多");
}
// 使用上次的时间戳,但带上回拨标记
timestamp = lastTimestamp;
} else { // 时钟正常,重置回拨计数器
backwardCounter = 0L;
}
// ... 生成ID,将backwardCounter也编码进去
}
}
public class SnowflakeWithFallback { // 当时钟回拨过大时,使用随机数生成器兜底
private final Random random = new Random();
public long nextId() { try { return snowflakeNextId();
} catch (ClockBackwardException e) { // 当时钟回拨无法处理时,使用随机ID兜底
log.warn("时钟回拨,使用随机ID兜底", e); return generateRandomId();
}
}
private long generateRandomId() { // 生成一个基于随机数的ID,但保证不会与正常ID冲突
// 方法:最高位置1,标识这是兜底ID
long randomId = random.nextLong() & Long.MAX_VALUE; return randomId | (1L << 63); // 最高位置1
}
}
| 方案 | 优点 | 缺点 | 适用场景 |
|---|---|---|---|
| 等待时钟 | 保持ID连续性 | 可能阻塞线程 | 回拨小的场景(<5ms) |
| 回拨计数器 | 不阻塞线程 | ID不连续 | 频繁小回拨场景 |
| 随机数兜底 | 保证可用性 | ID可能重复 | 紧急情况备用 |
假如公司有300多台服务器,但雪花算法只支持1024个机器ID。
更糟糕的是,有次扩容时, 两台机器配了相同的workerId,导致生成的ID大量重复。
机器ID分配为什么难?
@Componentpublic class WorkerIdAssigner {
@Autowired
private JdbcTemplate jdbcTemplate;
private Long workerId;
@PostConstruct
public void init() { // 尝试获取或分配workerId
this.workerId = assignWorkerId();
}
private Long assignWorkerId() { String hostname = getHostname(); String ip = getLocalIp();
// 查询是否已分配
String sql = "SELECT worker_id FROM worker_assign WHERE hostname = ? OR ip = ?";
List<Long> existingIds = jdbcTemplate.queryForList(sql, Long.class, hostname, ip);
if (!existingIds.isEmpty()) { return existingIds.get(0);
}
// 分配新的workerId
for (int i = 0; i < 1024; i++) { try {
sql = "INSERT INTO worker_assign (worker_id, hostname, ip, created_time) VALUES (?, ?, ?, NOW())"; int updated = jdbcTemplate.update(sql, i, hostname, ip); if (updated > 0) {
log.info("分配workerId成功: {} -> {}:{}", i, hostname, ip); return (long) i;
}
} catch (DuplicateKeyException e) { // workerId已被占用,尝试下一个
continue;
}
}
throw new RuntimeException("没有可用的workerId");
}
// 心跳保活
@Scheduled(fixedDelay = 30000)
public void keepAlive() { if (workerId != null) { String sql = "UPDATE worker_assign SET last_heartbeat = NOW() WHERE worker_id = ?";
jdbcTemplate.update(sql, workerId);
}
}
@PreDestroy
public void cleanup() { // 应用关闭时释放workerId(可选)
if (workerId != null) { String sql = "DELETE FROM worker_assign WHERE worker_id = ?";
jdbcTemplate.update(sql, workerId);
}
}
}
public class ZkWorkerIdAssigner {
private CuratorFramework client; private String workerPath = "/snowflake/workers"; private Long workerId;
public Long assignWorkerId() throws Exception { // 创建持久化节点
client.create().creatingParentsIfNeeded().forPath(workerPath);
// 创建临时顺序节点
String sequentialPath = client.create()
.withMode(CreateMode.EPHEMERAL_SEQUENTIAL)
.forPath(workerPath + "/worker-");
// 提取序号作为workerId
String sequenceStr = sequentialPath.substring(sequentialPath.lastIndexOf('-') + 1); long sequence = Long.parseLong(sequenceStr);
// 序号对1024取模得到workerId
this.workerId = sequence % 1024;
// 监听节点变化,如果连接断开自动释放
client.getConnectionStateListenable().addListener((curator, newState) -> { if (newState == ConnectionState.LOST || newState == ConnectionState.SUSPENDED) {
log.warn("ZK连接异常,workerId可能失效: {}", workerId);
}
});
return workerId;
}
}
public class IpBasedWorkerIdAssigner {
// 10位workerId,可以拆分为:3位机房 + 7位机器
private static final long DATACENTER_BITS = 3L; private static final long WORKER_BITS = 7L;
public long getWorkerId() { try { String ip = getLocalIp();
String[] segments =gov.zuqiu.zc.jrzb1.info ip.split("\\.");
// 使用IP后两段计算workerId
int third = Integer.parseInt(segments[2]); // 0-255
int fourth = Integer.parseInt(segments[3]); // 0-255
// 机房ID:取第三段的低3位 (0-7)
long datacenterId = third & ((1 << DATACENTER_BITS) - 1);
// 机器ID:取第四段的低7位 (0-127)
long workerId = fourth & ((1 << WORKER_BITS) - 1);
// 合并:3位机房 + 7位机器 = 10位workerId
return (datacenterId << WORKER_BITS) | workerId;
} catch (Exception e) { // 降级方案:使用随机数,但设置标志位
log.warn("IP计算workerId失败,使用随机数", e); return new Random().nextInt(1024) | (1L << 9); // 最高位置1表示随机
}
}
}
| 方案 | 优点 | 缺点 | 适用场景 |
|---|---|---|---|
| 数据库分配 | 精确控制 | 依赖DB,有单点风险 | 中小规模固定集群 |
| ZK分配 | 自动故障转移 | 依赖ZK,复杂度高 | 大规模动态集群 |
| IP计算 | 简单无依赖 | IP可能冲突 | 网络规划规范的场景 |
我们的订单服务在促销期间, 单机QPS达到5万,经常出现“序列号耗尽”的警告日志。
虽然雪花算法理论上支持4096/ms的序列号,但实际使用中发现,在高并发下还是可能不够用。
序列号为什么可能耗尽?
public class MicrosecondSnowflake { // 调整位分配:使用微秒级时间戳
// 1位符号位 + 36位微秒时间戳 + 10位机器ID + 17位序列号
private static final long TIMESTAMP_BITS = 36L; // 微秒时间戳
private static final long SEQUENCE_BITS = 17L; // 13万/微秒
private long lastMicroTimestamp = -1L; private long sequence = 0L;
public synchronized long nextId() { long currentMicros = getCurrentMicroseconds();
if (currentMicros < lastMicroTimestamp) { // 处理时钟回拨
throw new RuntimeException("时钟回拨");
}
if (currentMicros == lastMicroTimestamp) {
sequence = (sequence + 1) & ((1 << SEQUENCE_BITS) - 1); if (sequence == 0) { // 等待下一个微秒
currentMicros = waitNextMicros(lastMicroTimestamp);
}
} else {
sequence = 0L;
}
lastMicroTimestamp = currentMicros;
return ((currentMicros) << (SEQUENCE_BITS + WORKER_BITS)) |
(workerId << SEQUENCE_BITS) |
sequence;
}
private long getCurrentMicroseconds() { // 获取微秒级时间戳
return System.currentTimeMillis() * 1000 +
(System.nanoTime() / 1000 % 1000);
}
}
public class SegmentedSequenceSnowflake { // 为不同的业务类型分配不同的序列号段
private Map<String, Long> sequenceMap = new ConcurrentHashMap<>();
public long nextId(String businessType) { long timestamp = System.currentTimeMillis();
// 获取该业务类型的序列号
Long lastTimestamp = sequenceMap.get(businessType + "_ts"); Long sequence = sequenceMap.get(businessType);
if (lastTimestamp == null || lastTimestamp != timestamp) { // 新的毫秒,重置序列号
sequence = 0L;
sequenceMap.put(businessType + "_ts", timestamp);
} else { // 同一毫秒内,递增序列号
sequence = sequence + 1; if (sequence >= 4096) { // 等待下一个毫秒
timestamp = waitNextMillis(timestamp);
sequence = 0L;
sequenceMap.put(businessType + "_ts", timestamp);
}
}
sequenceMap.put(businessType, sequence);
// 将业务类型编码到workerId中
long businessWorkerId = encodeBusinessType(workerId, businessType);
return ((timestamp - twepoch) << timestampShift) |
(businessWorkerId << workerIdShift) |
sequence;
}
private long encodeBusinessType(long baseWorkerId, String businessType) { // 使用workerId的高几位表示业务类型
int typeCode = businessType.hashCode() & 0x1F; // 5位,32种业务
return (typeCode << 5) | (baseWorkerId & 0x1F);
}
}
public class IdPoolSnowflake { // 预生成ID池,缓解瞬时压力
private BlockingQueue<Long> idQueue = new LinkedBlockingQueue<>(10000); private volatile boolean isGenerating = false;
// 后台线程预生成ID
private Thread generatorThread = new Thread(() -> { while (!Thread.currentThread().isInterrupted()) { try { if (idQueue.size() < 5000 && !isGenerating) {
isGenerating = true;
generateBatchIds(1000);
isGenerating = false;
}
Thread.sleep(1); // 短暂休眠
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
});
public IdPoolSnowflake() {
generatorThread.setDaemon(true);
generatorThread.start();
}
public long nextId(mip.euakj.com) { try { // 从队列中获取预生成的ID
Long id = idQueue.poll(10, TimeUnit.MILLISECONDS); if (id != null) { return id;
}
// 队列为空,同步生成
log.warn("ID队列空,同步生成ID"); return snowflake.nextId();
} catch (InterruptedException e) {
Thread.currentThread().interrupt(); return snowflake.nextId();
}
}
private void generateBatchIds(int count) { for (int i = 0; i < count; i++) { try {
idQueue.put(snowflake.nextId());
} catch (InterruptedException e) {
Thread.currentThread().interrupt(); break;
}
}
}
}

雪花算法的41位时间戳能表示多少时间?
2^41 / 1000 / 60 / 60 / 24 / 365 ≈ 69年
看起来很长?
但这里有个大坑: 起始时间的选择。
如果起始时间设置不当,系统可能很快就面临时间戳溢出问题。
时间戳为什么可能溢出?
public class SnowflakeWithCustomEpoch { // 自定义起始时间:2020-01-01 00:00:00
private static final long CUSTOM_EPOCH = 1577836800000L; // 2020-01-01
// 计算剩余可用时间
public void checkRemainingTime() { long maxTimestamp = (1L << 41) - 1; // 41位最大时间戳
long currentTime = System.currentTimeMillis();
long elapsed = currentTime - CUSTOM_EPOCH; long remaining = maxTimestamp - elapsed;
long remainingYears = remaining / 1000 / 60 / 60 / 24 / 365;
log.info("雪花算法剩余可用时间: {}年 ({}毫秒)",
remainingYears, remaining);
if (remainingYears < 5) {
log.warn("雪花算法将在{}年后溢出,请准备升级方案", remainingYears);
}
}
public long nextId() { long timestamp = System.currentTimeMillis() - CUSTOM_EPOCH;
if (timestamp > maxTimestamp) { throw new RuntimeException("时间戳溢出,请升级ID生成方案");
}
// ... 生成ID
return (timestamp << timestampShift) |
(workerId << workerIdShift) |
sequence;
}
来自 “ ITPUB博客 ” ,链接:https://blog.itpub.net/70048957/viewspace-3107278/ ,如需转载,请注明出处,否则将追究法律责任。