DRPC-Proxy - 基於storm DRPC的遠程調用

DRPC-Proxy 簡介

DRPC-Proxy是基於使用storm的DRPC做RPC服務，解耦業務代碼與storm框架代碼的微框架；在使用某些場景下，有使用DRPC但不注重使用storm的流式計算的需求，通常情況下使用DRPCServer做爲服務提供方接收請求，bolt中處理業務，ReturnResults返回結果，勢必在bolt中會將業務代碼與storm代碼交織、耦合，爲後期升級、重構、擴展留下難題。DRPC-Proxy是提供解耦的框架，服務消費方使用動態代理生調用DRPCClient與DRPCServer通訊，DRPCServer將請求匹配到對於的服務提供方的實現，最終結果由DRPCServer返回給消費方。

源碼：https://github.com/zrk1000/drpcproxy

DRPC-Proxy 特點

• 解耦storm與業務代碼，開發過程中對storm無感知
• 使用簡單，導入jar包，properties中添加相關服務的配置，pom.xml中添加依賴及profile
• 支持三種模式開發，脫離storm進行業務開發-rely模式，LocalDRPC模式，Remote模式
• 提供spring環境下的支持，無spring亦可
• 異常可遠程拋出
• 對DRPC無封裝，使用原生代碼調用
• 集成AKKA，保證單線程下bolt對高併發的支持

Module 說明

proxy : 基於接口的drpc調用

proxy-spring : 支持spring環境的基於接口的drpc調用

demo：

demo-customer : 服務消費者

demo-customer-spring : spring環境服務消費者

demo-server : 服務接口

demo-serviceimpl : 服務提供者

demo-serviceimpl-spring : spring環境服務提供者

用法1(非spring環境)

服務接口API

 public interface UserService {
    User getUser(String name) throws MyException;
}

服務提供者

public class UserServiceImpl implements UserService {
    public User getUser(String name) throws MyException{
        User user = new User();
        if("tom".equals(name)){
            user.setAge(12);
            user.setId(111L);
            user.setName("tom");
        }else {
            throw new MyOnlyException("業務異常");
        }
        return user;
    }

drpcproxy-provider.properties

service.impls=\
#  com.zrk1000.demo.serviceimpl.TestServiceImpl,\
  com.zrk1000.demo.serviceimpl.UserServiceImpl
drpc.spout.num=1
drpc.dispatch.bolt.num=1
drpc.result.bolt.num=1
drpc.spout.name=spout_name
drpc.topology.name=topology_name

啓動腳本

storm jar provider.jar  com.zrk1000.proxy.ConfigMain drpcSpoutName topologyName

服務消費者

public class Runner {
  public static void main(String[] args) {
      ServiceImplFactory.init();
      UserService userService = ServiceImplFactory.newInstance(UserService.class);
      User user = userService.getUser("tom");
      System.out.println("------------user:"+user.toString());
  }
}

drpcproxy-consumer.properties

drpc.client.config.storm.thrift.transport=org.apache.storm.security.auth.SimpleTransportPlugin
drpc.client.config.storm.nimbus.retry.times=3
drpc.client.config.storm.nimbus.retry.interval.millis=10000
drpc.client.config.storm.nimbus.retry.intervalceiling.millis=60000
drpc.client.config.drpc.max_buffer_size=104857600
drpc.client.host=192.168.1.81
drpc.client.port=3772
drpc.client.timeout=50000

topology.mapping.config.zrk1000-service-provider=\
#    com.zrk1000.demo.service.TestService,\
    com.zrk1000.demo.service.UserService

#topology.mapping.config.zrk1000-service-provider-spring=\
#    com.zrk1000.demo.service.UserService

#remote,local,rely
drpc.pattern=${profiles.pattern}

用法2(spring環境-springboot)

服務接口API

 public interface UserService {
    UserDto getUser(Long id);
}

服務提供者

@Service
@Transactional
public class UserServiceImpl implements UserService {
    @Autowired
    private UserRepository userRepository;

    public UserDto getUser(Long id) {
        return convert(userRepository.findOne(id));
    }

}

drpcproxy-provider.properties

#業務所在的包名，使用AnnotationConfigApplicationContext創建spring上下文環境 ，建議使用springboot，可支持基於xml構建上下文
service.impls=com.zrk1000.demo
drpc.spout.num=1
drpc.dispatch.bolt.num=1
drpc.result.bolt.num=1
drpc.spout.name=spout_name
drpc.topology.name=topology_name

啓動腳本

storm jar provider.jar  com.zrk1000.proxy.SpringMain drpcSpoutName topologyName

服務消費者

@RestController
@RequestMapping("user")
public class UserController {
    @Autowired
    private UserService userService;

    @RequestMapping(value = "/{id}",method = RequestMethod.GET)
    public UserDto getUser(@PathVariable("id") Long id){
        return userService.getUser(id);
    }
}

drpcproxy-consumer.properties

drpc.client.config.storm.thrift.transport=org.apache.storm.security.auth.SimpleTransportPlugin
drpc.client.config.storm.nimbus.retry.times=3
drpc.client.config.storm.nimbus.retry.interval.millis=10000
drpc.client.config.storm.nimbus.retry.intervalceiling.millis=60000
drpc.client.config.drpc.max_buffer_size=104857600
drpc.client.host=192.168.1.81
drpc.client.port=3772
drpc.client.timeout=5000

topology.mapping.config.zrk1000-service-provider-spring=\
#  com.zrk1000.demo.service.GroupService,\
  com.zrk1000.demo.service.UserService

StormConfig

@Profile({"local","remote"})
@Configuration
@ServiceScan(basePackages = "com.zrk1000.demo.service",
//        excludeClasses = {UserService.class},
        rpcHandleBeanRef="stormDrpcHandle")
public class StormConfig {

    @Profile("local")
    @Scope("singleton")
    @Bean("stormDrpcHandle")
    public RpcHandle getStormLocalRpcHandle(){
        StormLocalDrpcHandle drpcHandle = null;
        try {
            Set<String> serviceImpls = ServiceImplFactory.loadServiceImpls();
            SpringBoltHandle springBoltHandle = new SpringBoltHandle(serviceImpls.toArray(new String[serviceImpls.size()]));
            drpcHandle = new StormLocalDrpcHandle(springBoltHandle);
        } catch (IOException e) {
            throw new RuntimeException("初始化stormDrpcHandle失敗");
        }
        return  drpcHandle;
    }

    @Bean
    @ConfigurationProperties("drpc.client")
    public DrpcClientConfig getDrpcClientConfig(){
        return new DrpcClientConfig();
    }

    @Bean
    @ConfigurationProperties("topology.mapping")
    public TopologyMapping getTopologyMapping(){
        return new TopologyMapping();
    }

    @Profile("remote")
    @Bean("stormDrpcHandle")
    public RpcHandle getStormRemoteRpcHandle(DrpcClientConfig clientConfig,TopologyMapping topologyMapping){
        Config config = new Config();
        config.putAll(clientConfig.getConfig());
        return  new StormRemoteDrpcHandle(config,clientConfig.getHost(),clientConfig.getPort(),clientConfig.getTimeout(),topologyMapping.getConfig());
    }

    class DrpcClientConfig{
        private String host;
        private Integer port;
        private Integer timeout;
        private Map<String,String> config ;
        //getter setter
    }

    class TopologyMapping {
        Map<String, Set<String>> config;
        //getter setter
    }

}

源碼：https://github.com/zrk1000/drpcproxy