阿里开源多语言序列化框架 Fury

蚂蚁集团多语言序列化框架 Fury 于 2023 年 7 月份正式开源，2023 年 12 月 15 Fury 捐赠给 Apache 软件基金会 (Apache Software Foundation)。Fury 以 14 个约束性投票 (binding votes)，6 个非约束性投票 (non-binding votes)，无弃权和反对票，全票通过投票决议的优秀表现，正式加入 Apache 孵化器，开启新的旅程。

❝

项目地址: https://github.com/apache/incubator-fury
官网: https://furyio.org/
Apache : https://fury.apache.org/ https://incubator.apache.org/clutch/fury.html

❞

Fury 简介

Fury 是一个基于 JIT 动态编译和零拷贝技术的多语言序列化框架，支持 Java/Python/C++/JavaScript/Golang/ Scala/Rust 等语言，提供最高 170 倍的性能和极致的易用性，可以大幅提升大数据计算、AI 应用和微服务等系统的性能，降低服务间调用延迟，节省成本并提高用户体验；同时提供全新的多语言编程范式，解决现有的多语言序列化动态性不足的问题，显著降低多语言系统的研发成本，帮助业务快速迭代。

Fury 特性

• 支持主流编程语言如Java/Python/C++/Golang，其它语言可轻易扩展；
• 多语言/跨语言自动序列化任意对象，无需创建IDL文件、手动编译schema生成代码以及将对象转换为中间格式；
• 多语言/跨语言自动序列化共享引用和循环引用，用户只需要关心对象，不需要关心数据重复或者递归错误；
• 基于JIT动态编译技术在运行时自动生成序列化代码优化性能，增加方法内联、代码缓存和死代码消除，减少虚方法调用/条件分支/Hash查找/元数据写入等，提供相比其它序列化框架20~200倍以上的性能；
• Zero-Copy序列化支持，支持Out of band序列化协议，支持堆外内存读写；
• 提供缓存友好的二进制随机访问行存格式，支持跳过序列化和部分序列化，并能和列存自动互转;
• 无缝替代JDK/Kryo/Hessian等Java序列化框架，无需修改任何代码，同时提供相比Kryo 20倍以上的性能，相比Hessian100倍以上的性能，相比JDK自带序列化200倍以上的性能，可以大幅提升高性能场景RPC调用和对象持久化效率；
• 支持共享引用和循环引用的Golang序列化框架；
• 支持对象自动序列化的Golang序列化框架；

Fury 安装

Java

Nightly snapshot:

<repositories>
<repository>
<id>apache</id>
<url>https://repository.apache.org/snapshots/</url>
<releases>
<enabled>false</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
</repositories>
<dependency>
<groupId>org.apache.fury</groupId>
<artifactId>fury-core</artifactId>
<version>0.5.0-SNAPSHOT</version>
</dependency>
<!-- row/arrow format support -->
<!-- <dependency>
  <groupId>org.apache.fury</groupId>
  <artifactId>fury-format</artifactId>
  <version>0.5.0-SNAPSHOT</version>
</dependency> -->

Release version:

<dependency>
<groupId>org.furyio</groupId>
<artifactId>fury-core</artifactId>
<version>0.4.1</version>
</dependency>
<!-- row/arrow format support -->
<!-- <dependency>
  <groupId>org.furyio</groupId>
  <artifactId>fury-format</artifactId>
  <version>0.4.1</version>
</dependency> -->

Scala

libraryDependencies += "org.furyio" % "fury-core" % "0.4.1"

Python

# Python wheel will be released in the future.
# Currently you need to specify `--pre` to install
# the unstable version.
pip install pyfury

Golang

go get https://github.com/apache/incubator-fury/go/fury

JavaScript

npm install @furyjs/fury

Rust

# Cargo.toml

[dependencies]
fury = { git= "https://github.com/apache/incubator-fury.git", branch = "main" }
fury_derive = { git= "https://github.com/apache/incubator-fury.git", branch = "main" }
lazy_static = { version = "1.4.0" }

Fury 使用

Java

import java.util.List;
import java.util.Arrays;
import io.fury.*;

public class Example {
public static void main(String[] args) {
    SomeClass object = new SomeClass();
// Note that Fury instances should be reused between
// multiple serializations of different objects.
    Fury fury = Fury.builder().withLanguage(Language.JAVA)
// Allow to deserialize objects unknown types,
// more flexible but less secure.
// .withSecureMode(false)
      .build();
// Registering types can reduce class name serialization overhead, but not mandatory.
// If secure mode enabled, all custom types must be registered.
    fury.register(SomeClass.class);
byte[] bytes = fury.serialize(object);
    System.out.println(fury.deserialize(bytes));
  }
}

Python

from typing import Dict
import pyfury

class SomeClass:
    f1: "SomeClass"
    f2: Dict[str, str]
    f3: Dict[str, str]

fury = pyfury.Fury(ref_tracking=True)
fury.register_class(SomeClass, "example.SomeClass")
obj = SomeClass()
obj.f2 = {"k1": "v1", "k2": "v2"}
obj.f1, obj.f3 = obj, obj.f2
data = fury.serialize(obj)
# bytes can be data serialized by other languages.
print(fury.deserialize(data))

Golang

package main

import furygo "https://github.com/apache/incubator-fury/go/fury"
import "fmt"

func main() {
	type SomeClass struct {
		F1 *SomeClass
		F2 map[string]string
		F3 map[string]string
	}
	fury := furygo.NewFury(true)
	if err := fury.RegisterTagType("example.SomeClass", SomeClass{}); err != nil {
		panic(err)
	}
	value := &SomeClass{F2: map[string]string{"k1": "v1", "k2": "v2"}}
	value.F3 = value.F2
	value.F1 = value
	bytes, err := fury.Marshal(value)
	if err != nil {
	}
	var newValue interface{}
	// bytes can be data serialized by other languages.
	if err := fury.Unmarshal(bytes, &newValue); err != nil {
		panic(err)
	}
	fmt.Println(newValue)
}

JavaScript

import Fury, { Type } from '@furyjs/fury';

/**
 * @furyjs/hps use v8's fast-calls-api that can be called directly by jit, ensure that the version of Node is 20 or above.
 * Experimental feature, installation success cannot be guaranteed at this moment
 * If you are unable to install the module, replace it with `const hps = null;`
 **/
import hps from '@furyjs/hps';

// Now we describe data structures using JSON, but in the future, we will use more ways.
const description = Type.object('example.foo', {
  foo: Type.string(),
});
const fury = new Fury({ hps });
const { serialize, deserialize } = fury.registerSerializer(description);
const input = serialize({ foo: 'hello fury' });
const result = deserialize(input);
console.log(result);

Rust

use fury::{from_buffer, to_buffer};
use fury_derive::{Deserialize, FuryMeta, Serialize};

#[derive(FuryMeta, Deserialize, Serialize, Debug, PartialEq)]
#[tag("example.foo")]
struct Animal {
    name: String,
    category: String,
}

#[derive(FuryMeta, Deserialize, Serialize, Debug, PartialEq)]
#[tag("example.bar")]
struct Person {
    name: String,
    age: u32,
    pets: Vec<Animal>,
}

fn main() {
    let penson = Person {
        name: "hello".to_string(),
        age: 12,
        pets: vec![
            Animal {
                name: "world1".to_string(),
                category: "cat".to_string(),
            },
            Animal {
                name: "world2".to_string(),
                category: "dog".to_string(),
            },
        ],
    };
    let bin = to_buffer(&penson);
    let obj: Person = from_buffer(&bin).expect("should success");
    assert_eq!(obj, penson);
}

欢迎加入 DevXTalk 的技术交流群,在这里你可以和大佬们一起交流分享技术,行业经验,提升自我。总之期待你的加入...

「看完本文有收获？转发给更多的朋友吧」

「DevXTalk 不止于技术」