Flink(5)讀取數據源的幾種方式

執行環境

創建一個執行環境,表示當前執行程序的上下文,如果程序是獨立調用的則此方法返回本地執行環境,如果從命令行客戶端調用程序提交到集群,則此方法返回此集群的執行環境,也就是是說,getExecutionEnvironment(),會根據查詢運行的方式決定返回什么樣的運行環境。是最常用的一種創建執行環境的方式。

# 流處理環境
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
#批處理
ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();

1.讀取的數據源-Elements

DataStreamSource<Integer> integerDataStreamSource = env.fromElements(1, 2, 2, 5, 44);

2.讀取的數據源-Collection

public class SensorReading {

    private String id;
    private Long timestamp;
    private Double temperature;

    public SensorReading() {
    }

    public SensorReading(String id, Long timestamp, Double temperature) {
        this.id = id;
        this.timestamp = timestamp;
        this.temperature = temperature;
    }

    public String getId() {
        return id;
    }

    public void setId(String id) {
        this.id = id;
    }

    public Long getTimestamp() {
        return timestamp;
    }

    public void setTimestamp(Long timestamp) {
        this.timestamp = timestamp;
    }

    public Double getTemperature() {
        return temperature;
    }

    public void setTemperature(Double temperature) {
        this.temperature = temperature;
    }

    @Override
    public String toString() {
        return "SensorReading{" +
                "id='" + id + '\'' +
                ", timestamp=" + timestamp +
                ", temperature=" + temperature +
                '}';
    }
DataStreamSource<SensorReading> sensorDataStream = env.fromCollection(Arrays.asList(
                new SensorReading("sensor_1", 1547718199L, 35.8),
                new SensorReading("sensor_6", 1547718201L, 15.8),
                new SensorReading("sensor_7", 1547718202L, 37.8),
                new SensorReading("sensor_10", 1547718203L, 38.8)
        ));

3.讀取的數據源-File

 //從文件讀取
DataStreamSource<String> stringDataStreamSource = env.readTextFile("/path/to/hello.txt");

4.從socket文本流讀取數據

DataStream<String> inputDataStream = env.socketTextStream(localhost,7777);

最終執行

 sensorDataStream.print("sensor");
 integerDataStreamSource.print("int");
 stringDataStreamSource.print("file");

 env.execute("my job name");
?著作權歸作者所有,轉載或內容合作請聯系作者
平臺聲明:文章內容(如有圖片或視頻亦包括在內)由作者上傳并發布,文章內容僅代表作者本人觀點,簡書系信息發布平臺,僅提供信息存儲服務。

推薦閱讀更多精彩內容