springCloud-ribbon 源碼分析

源碼閱讀入口 MATE-INF/spring.factories

org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
org.springframework.cloud.netflix.ribbon.RibbonAutoConfiguration

飢餓加載

每個service服務調用其它服務時, 會爲每個請求目標創建各自的Client, 並會以serviceId爲 key, Client實例作爲value, 存儲在ConcurrenthashMap中, 默認這些client會在第一次調用目標服務時創建(懶加載), 但是會發生, 某些時候會在調用第一次目標服務時超時, 我們可以通過配置修改ribbonClient加載模式爲飢餓加載

#對全部ribbonClient都啓動飢餓加載配置
ribbon.eager-load.enabled = true //啓動飢餓加載
#對某些特定service的ribbonClient 啓動飢餓加載
ribbon.eager-load.clients= service1, service2, ...

源碼:

	
	/**
	 * ribbon 客戶端規範
	 */
	@Autowired(required = false)
	private List<RibbonClientSpecification> configurations = new ArrayList<>();

	/**
	 * 工廠設計模式
	 * spring 請求客戶端工廠,
	 */
	@Bean
	public SpringClientFactory springClientFactory() {
		SpringClientFactory factory = new SpringClientFactory();
		factory.setConfigurations(this.configurations);
		return factory;
	}

查看代碼中有這個配置, 猜想 ribbonClient是通過這個SpringClientFactory實例創建的

public class SpringClientFactory extends NamedContextFactory<RibbonClientSpecification> {
	/**
	 * 通過name獲取restClient
	 * @param name 作爲搜索條件
	 * @param clientClass 是client的實例類型
	 */
	public <C extends IClient<?, ?>> C getClient(String name, Class<C> clientClass) {
		return getInstance(name, clientClass);
	}

	@Override
	public <C> C getInstance(String name, Class<C> type) {
		C instance = super.getInstance(name, type);
		if (instance != null) {
			return instance;
		}
		IClientConfig config = getInstance(name, IClientConfig.class);
		return instantiateWithConfig(getContext(name), type, config);
	}
}

public abstract class NamedContextFactory<C extends NamedContextFactory.Specification> implements DisposableBean, ApplicationContextAware {
    private Map<String, AnnotationConfigApplicationContext> contexts = new ConcurrentHashMap();
    
	public <T> T getInstance(String name, Class<T> type) {
        AnnotationConfigApplicationContext context = this.getContext(name);
        return BeanFactoryUtils.beanNamesForTypeIncludingAncestors(context, type).length > 0 ? context.getBean(type) : null;
    }
    //從context中獲取獲取restClient
    protected AnnotationConfigApplicationContext getContext(String name) {
        //如果不包含這個key, 這裏通過name爲key 創建restClient環境, 然後再在contex中緩存一份, 將restClient返回. 這裏用了雙檢鎖. 
        if (!this.contexts.containsKey(name)) {
            synchronized(this.contexts) {
                if (!this.contexts.containsKey(name)) {
                    this.contexts.put(name, this.createContext(name));
                }
            }
        }

        return (AnnotationConfigApplicationContext)this.contexts.get(name);
    }

}

以上代碼能表示:

所有的restClient 通過name爲key, 存儲在ConcurrentHashMap 中, 並且是在使用時才創建加載, 這裏顯示的restClient 是懶加載, name應該是各個爲服務的name屬性, 存取屬於線程安全的,

再回到RibbonAutoConfiguration,查看飢餓加載配置

	/**
	 * ribbon飢餓加載屬性
	 */
	@Autowired
	private RibbonEagerLoadProperties ribbonEagerLoadProperties;

	/**
	 * ribbon的飢餓加載設置爲true, 啓動時就會初始化ribbon的上下文環境
	 */
	@Bean
	@ConditionalOnProperty("ribbon.eager-load.enabled")
	public RibbonApplicationContextInitializer ribbonApplicationContextInitializer() {
		return new RibbonApplicationContextInitializer(springClientFactory(),
				ribbonEagerLoadProperties.getClients());
	}

@ConfigurationProperties(prefix = "ribbon.eager-load")
public class RibbonEagerLoadProperties {

	private boolean enabled = false;

	private List<String> clients;

	public boolean isEnabled() {
		return enabled;
	}

	public void setEnabled(boolean enabled) {
		this.enabled = enabled;
	}

	public List<String> getClients() {
		return clients;
	}

	public void setClients(List<String> clients) {
		this.clients = clients;
	}

}

RibbonEagerLoadProperties作爲ribbon的飢餓加載配置屬性, 配置屬性的前綴是ribbon.eager-load, 屬性有enabled和 clients, 默認飢餓加載不生效(生效懶加載 )

public class RibbonApplicationContextInitializer
		implements ApplicationListener<ApplicationReadyEvent> {

	private final SpringClientFactory springClientFactory;

	// List of Ribbon client names
	private final List<String> clientNames;

	public RibbonApplicationContextInitializer(SpringClientFactory springClientFactory,
			List<String> clientNames) {
		this.springClientFactory = springClientFactory;
		this.clientNames = clientNames;
	}

	protected void initialize() {
		if (clientNames != null) {
			for (String clientName : clientNames) {
				this.springClientFactory.getContext(clientName);
			}
		}
	}

	@Override
	public void onApplicationEvent(ApplicationReadyEvent event) {
		initialize();
	}
}

看RibbonApplicationContextInitializer的構造器, clientNames 這個就是RibbonEagerLoadProperties 中的clients屬性, 並且RibbonApplicationContextInitializer是一個監聽器, 監聽的事件是ApplicationReadyEvent, 當應用加載完成, 調用onApplicationEvent 進行client初始化, 初始化調用了springClientFactory.getContext(), 那麼這時候會將配置的所有客戶端就加載到上面提到來的ConcurrentHashMap中

也就是說加載順序如下
- 如果發現配置了ribbon.eager-load.enabled, 就會加載 RibbonEagerLoadProperties
- 通過RibbonEagerLoadProperties 初始化RibbonApplicationContextInitializer, 並注入springClientFactory
- 監聽ApplicationReadyEvent事件表示應用啓動, 準備完成, 開始初始化各個應用的restClient, 並存入ConcurrentHashMap中,
至此, restClient 加載完畢

此部分用到的設計模式: 工廠設計模式, 監聽者模式

負載均衡

查看RibbonAutoConfiguration 中的自動配置

public class RibbonAutoConfiguration {	
	/**
	 * 負載均衡客戶端, 注入spring客戶端工廠
	 */
	@Bean
	@ConditionalOnMissingBean(LoadBalancerClient.class)
	public LoadBalancerClient loadBalancerClient() {
		return new RibbonLoadBalancerClient(springClientFactory());
	}
}

這裏是返回的spring cloud實現的LoadBalancerClient接口實現, 先看這個接口

public interface ServiceInstanceChooser {
    ServiceInstance choose(String serviceId);
}

public interface LoadBalancerClient extends ServiceInstanceChooser {
    <T> T execute(String serviceId, LoadBalancerRequest<T> request) throws IOException;

    <T> T execute(String serviceId, ServiceInstance serviceInstance, LoadBalancerRequest<T> request) throws IOException;

    URI reconstructURI(ServiceInstance instance, URI original);
}

LoadBalancerClient接口與其繼承的ServiceInstanceChooser有三類抽象方法 execute reconstructURI和choose

choose選擇所使用的service
reconstructURI通過選擇的service, 獲取到相應的host, port等信息重構URI
execute執行uri發送請求

查看choose在RibbonLoadBalancerClient實現

public class RibbonLoadBalancerClient implements LoadBalancerClient {
	@Override
	public ServiceInstance choose(String serviceId) {
		return choose(serviceId, null);
	}
    
    /**
	 * New: Select a server using a 'key'.
	 * @param serviceId of the service to choose an instance for
	 * @param hint to specify the service instance
	 * @return the selected {@link ServiceInstance}
	 */
	public ServiceInstance choose(String serviceId, Object hint) {
		Server server = getServer(getLoadBalancer(serviceId), hint);
		if (server == null) {
			return null;
		}
		return new RibbonServer(serviceId, server, isSecure(server, serviceId),
				serverIntrospector(serviceId).getMetadata(server));
	}
    
	protected ILoadBalancer getLoadBalancer(String serviceId) {
		return this.clientFactory.getLoadBalancer(serviceId);
	}
}

上面的代碼是通過傳入的serviceId查詢ServiceInstance, 下面獲取服務時調用了

	protected ILoadBalancer getLoadBalancer(String serviceId) {
		return this.clientFactory.getLoadBalancer(serviceId);
	}

這個getLoadBancer()是初始化加載時的ClientFactory的行爲, 根據serviceId爲key存儲在ConcurrentHashMap中的數據返回服務, 這就與之前的ribbonClient關聯上了

在LoadBalancerClient接口同目錄下查看到一個比較熟悉的名字:LoadBalanced註解, 不難想象, 我們在直接使用ribbon時, 就是用這個註解爲spring的restTemplate增加負載均衡功能的, 所以認定我們源碼閱讀的路線還是表正確的!

同目錄下看到有LoadBalancerAutoConfiguration 這個應該是負載均衡自動配置了, 往下看

public class LoadBalancerAutoConfiguration {
        @ConditionalOnMissingClass({"org.springframework.retry.support.RetryTemplate"})
    static class LoadBalancerInterceptorConfig {
        LoadBalancerInterceptorConfig() {
        }

        @Bean
        public LoadBalancerInterceptor ribbonInterceptor(LoadBalancerClient loadBalancerClient, LoadBalancerRequestFactory requestFactory) {
            return new LoadBalancerInterceptor(loadBalancerClient, requestFactory);
        }

        @Bean
        @ConditionalOnMissingBean
        public RestTemplateCustomizer restTemplateCustomizer(final LoadBalancerInterceptor loadBalancerInterceptor) {
            return (restTemplate) -> {
                List<ClientHttpRequestInterceptor> list = new ArrayList(restTemplate.getInterceptors());
                list.add(loadBalancerInterceptor);
                restTemplate.setInterceptors(list);
            };
        }
    }
}

自動配置中有這麼個內部類, 這個內部類中最後的那個配置方法是對restTemplate增加了一個攔截器, 這個攔截器是AsyncClientHttpRequestInterceptor, 顧名思義. 負載均衡攔截器

查看攔截器源碼

public class LoadBalancerInterceptor implements ClientHttpRequestInterceptor {
    private LoadBalancerClient loadBalancer;
    private LoadBalancerRequestFactory requestFactory;

    public LoadBalancerInterceptor(LoadBalancerClient loadBalancer, LoadBalancerRequestFactory requestFactory) {
        this.loadBalancer = loadBalancer;
        this.requestFactory = requestFactory;
    }

    public LoadBalancerInterceptor(LoadBalancerClient loadBalancer) {
        this(loadBalancer, new LoadBalancerRequestFactory(loadBalancer));
    }

    public ClientHttpResponse intercept(final HttpRequest request, final byte[] body, final ClientHttpRequestExecution execution) throws IOException {
        URI originalUri = request.getURI();
        String serviceName = originalUri.getHost();
        Assert.state(serviceName != null, "Request URI does not contain a valid hostname: " + originalUri);
        return (ClientHttpResponse)this.loadBalancer.execute(serviceName, this.requestFactory.createRequest(request, body, execution));
    }
}

驚喜呀, 攔截器中的攔截方法調用的就是LoadBalancerClient::execute方法. 這樣就回歸到了負載均衡客戶端執行請求的時刻了

我們再回顧LoadBalancerClient的 execute方法

public class RibbonLoadBalancerClient implements LoadBalancerClient {
    @Override
	public <T> T execute(String serviceId, LoadBalancerRequest<T> request)
			throws IOException {
		return execute(serviceId, request, null);
	}
    
    public <T> T execute(String serviceId, LoadBalancerRequest<T> request, Object hint)
			throws IOException {
		ILoadBalancer loadBalancer = getLoadBalancer(serviceId);
		Server server = getServer(loadBalancer, hint);
		if (server == null) {
			throw new IllegalStateException("No instances available for " + serviceId);
		}
		RibbonServer ribbonServer = new RibbonServer(serviceId, server,
				isSecure(server, serviceId),
				serverIntrospector(serviceId).getMetadata(server));

		return execute(serviceId, ribbonServer, request);
	}
    
    protected Server getServer(ILoadBalancer loadBalancer, Object hint) {
		if (loadBalancer == null) {
			return null;
		}
		// Use 'default' on a null hint, or just pass it on?
		return loadBalancer.chooseServer(hint != null ? hint : "default");
	}

最下面的這句

loadBalancer.chooseServer(hint != null ? hint : "default");

便是負載均衡算法的實現了

loadBalancer是ZoneAwareLoadBalancer的實例

最終是由這個實現的負載均衡, 通過serviceName查詢到所有service, 選擇其中一個service返回的過程, 詳細內容參考這裏

內部負載均衡是通過Irule實現的

ribbon提供了七種負載均衡規則

策略名	策略聲明	策略描述	實現說明
BestAvailableRule	public class BestAvailableRule extends ClientConfigEnabledRoundRobinRule	選擇一個最小的併發請求的server	逐個考察Server，如果Server被tripped了，則忽略，在選擇其中ActiveRequestsCount最小的server
AvailabilityFilteringRule	public class AvailabilityFilteringRule extends PredicateBasedRule	過濾掉那些因爲一直連接失敗的被標記爲circuit tripped的後端server，並過濾掉那些高併發的的後端server（active connections 超過配置的閾值）	使用一個AvailabilityPredicate來包含過濾server的邏輯，其實就就是檢查status裏記錄的各個server的運行狀態
WeightedResponseTimeRule	public class WeightedResponseTimeRule extends RoundRobinRule	根據相應時間分配一個weight，相應時間越長，weight越小，被選中的可能性越低。	一個後臺線程定期的從status裏面讀取評價響應時間，爲每個server計算一個weight。Weight的計算也比較簡單responsetime 減去每個server自己平均的responsetime是server的權重。當剛開始運行，沒有形成statas時，使用roubine策略選擇server.
RetryRule	public class RetryRule extends AbstractLoadBalancerRule	對選定的負載均衡策略機上重試機制。	在一個配置時間段內當選擇server不成功，則一直嘗試使用subRule的方式選擇一個可用的server
RoundRobinRule	public class RoundRobinRule extends AbstractLoadBalancerRule	roundRobin方式輪詢選擇server	輪詢index，選擇index對應位置的server
RandomRule	public class RandomRule extends AbstractLoadBalancerRule	隨機選擇一個server	在index上隨機，選擇index對應位置的server
ZoneAvoidanceRule	public class ZoneAvoidanceRule extends PredicateBasedRule	複合判斷server所在區域的性能和server的可用性選擇server	默認規則; 使用ZoneAvoidancePredicate和AvailabilityPredicate來判斷是否選擇某個server，前一個判斷判定一個zone的運行性能是否可用，剔除不可用的zone（的所有server），AvailabilityPredicate用於過濾掉連接數過多的Server。

ZoneAvoidanceRule是PredicateBasedRule的一個實現類，只不過這裏多一個過濾條件，ZoneAvoidanceRule中的過濾條件是以ZoneAvoidancePredicate爲主過濾條件和以AvailabilityPredicate爲次過濾條件組成的一個叫做CompositePredicate的組合過濾條件，過濾成功之後，繼續採用線性輪詢的方式從過濾結果中選擇一個出來

如下:

/**
 * 該類爲Ribbon的配置類
 * 該類不能被@ComponentScan掃描到
 * @author nicker
 * @description
 * @date 2018/5/16 18:25
 * @
 */
@Configuration
public class RibbonConfiguration {
 
    @Bean
    public IRule ribbonRule() {
        // 負載均衡規則，改爲隨機
        return new RandomRule();
    }
}

自定義負載均衡規則

繼承AbstractLoadBalancerRule抽象類或其子類實現

Server choose(Object var1);
public abstract void initWithNiwsConfig(IClientConfig clientConfig);

這兩個抽象方法

public class MyCustomerRule extends AbstractLoadBalancerRule
{
    @Override
    public Server choose(Object key)
    {
    	ILoadBalancer loadBalancer = getLoadBalancer();
        return loadBalancer.choose(Key);//實現獲取server的算法
    }

    @Override
    public void initWithNiwsConfig(IClientConfig clientConfig)
    {
        // TODO Auto-generated method stub

    }

}

配置生效:兩種

javaConfig
```
@Configuration
public class MyCustomerRuleConfiguration
{
    @Bean
    public IRule myCustomerRule()
    {
        return new MyCustomerRule();// 我自定義爲每臺機器5次
    }
}
```
注意: 在Ribbon的文檔中有這樣一段話：

The FooConfiguration has to be @Configuration but take care that it is not in a @ComponentScan for the main application context, otherwise it will be shared by all the @RibbonClients. If you use @ComponentScan (or @SpringBootApplication) you need to take steps to avoid it being included (for instance put it in a separate, non-overlapping package, or specify the packages to scan explicitly in the @ComponentScan).

大體意思是對於Ribbon的配置必須用@Configuration註解標識，並且不能被@Component註解或者@SpringBootApplication（因爲裏面包含了@Component）掃描到。因爲如果被@ComponetScan掃描到會導致所有的RibbonClient都去共享這個配置。

使用application配置文件

service-provider-user:
  ribbon:
    NFLoadBalancerRuleClassName: com.zcz.study.MyCustomerRule

springCloud-ribbon 源碼分析

飢餓加載

負載均衡

自定義負載均衡規則

推薦2款開源、美觀的WinForm UI控件庫

NET9 AspnetCore將整合OpenAPI的文檔生成功能而無需三方庫

在Linux下管理MySQL的大小寫敏感性

nginx+fastDFS+lua+graphicMagick部署

hadoop完全高可用配置

Hadoop MR Shuffle

springCloud-ribbon 源碼分析

hadoop hdfs NameNode 與 Secondary NameNode元數據操作

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結