diff --git a/data/connectors.js b/data/connectors.js index 86c05b82b834..76522eb02ab7 100644 --- a/data/connectors.js +++ b/data/connectors.js @@ -178,11 +178,5 @@ module.exports = [ longName: 'Test Data Generator source', type: 'Source', link: 'https://github.com/apache/pulsar/tree/master/pulsar-io/data-generator' - }, - { - name: 'twitter', - longName: 'Twitter Firehose source', - type: 'Source', - link: 'https://developer.twitter.com/en/docs' } ] diff --git a/docs/io-connectors.md b/docs/io-connectors.md index 1b94b6d6dd0f..cd355a9f9434 100644 --- a/docs/io-connectors.md +++ b/docs/io-connectors.md @@ -83,12 +83,6 @@ Pulsar has various source connectors, which are sorted alphabetically as below. * [Java class](https://github.com/apache/pulsar/blob/master/pulsar-io/flume/src/main/java/org/apache/pulsar/io/flume/FlumeConnector.java) -### Twitter firehose - -* [Configuration](io-twitter-source.md#configuration) - -* [Java class](https://github.com/apache/pulsar/blob/master/pulsar-io/twitter/src/main/java/org/apache/pulsar/io/twitter/TwitterFireHose.java) - ### Kafka * [Configuration](io-kafka-source.md#configuration) diff --git a/docs/io-develop.md b/docs/io-develop.md index 79fce31b031c..26714349c3cd 100644 --- a/docs/io-develop.md +++ b/docs/io-develop.md @@ -315,7 +315,7 @@ For Gradle users, there is a [Gradle Nar plugin available on the Gradle Plugin P :::tip -For more information about **how to use NAR for Pulsar connectors**, see {@inject: github:TwitterFirehose:/pulsar-io/twitter/pom.xml}. +For more information about **how to use NAR for Pulsar connectors**, see {@inject: github:DataGen:/pulsar-io/datagen/pom.xml}. ::: diff --git a/docs/io-quickstart.md b/docs/io-quickstart.md index bb7082b907bf..19474b0a6239 100644 --- a/docs/io-quickstart.md +++ b/docs/io-quickstart.md @@ -98,7 +98,7 @@ ls connectors **Example output** ```json - [{"name":"aerospike","description":"Aerospike database sink","sinkClass":"org.apache.pulsar.io.aerospike.AerospikeStringSink"},{"name":"cassandra","description":"Writes data into Cassandra","sinkClass":"org.apache.pulsar.io.cassandra.CassandraStringSink"},{"name":"kafka","description":"Kafka source and sink connector","sourceClass":"org.apache.pulsar.io.kafka.KafkaStringSource","sinkClass":"org.apache.pulsar.io.kafka.KafkaBytesSink"},{"name":"kinesis","description":"Kinesis sink connector","sinkClass":"org.apache.pulsar.io.kinesis.KinesisSink"},{"name":"rabbitmq","description":"RabbitMQ source connector","sourceClass":"org.apache.pulsar.io.rabbitmq.RabbitMQSource"},{"name":"twitter","description":"Ingest data from Twitter firehose","sourceClass":"org.apache.pulsar.io.twitter.TwitterFireHose"}] + [{"name":"aerospike","description":"Aerospike database sink","sinkClass":"org.apache.pulsar.io.aerospike.AerospikeStringSink"},{"name":"cassandra","description":"Writes data into Cassandra","sinkClass":"org.apache.pulsar.io.cassandra.CassandraStringSink"},{"name":"kafka","description":"Kafka source and sink connector","sourceClass":"org.apache.pulsar.io.kafka.KafkaStringSource","sinkClass":"org.apache.pulsar.io.kafka.KafkaBytesSink"},{"name":"kinesis","description":"Kinesis sink connector","sinkClass":"org.apache.pulsar.io.kinesis.KinesisSink"},{"name":"rabbitmq","description":"RabbitMQ source connector","sourceClass":"org.apache.pulsar.io.rabbitmq.RabbitMQSource"}] ``` If an error occurs when starting the Pulsar service, you may see an exception at the terminal running `pulsar/standalone`, diff --git a/docs/io-twitter-source.md b/docs/io-twitter-source.md deleted file mode 100644 index 092ae4792617..000000000000 --- a/docs/io-twitter-source.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -id: io-twitter-source -title: Twitter Firehose source connector -sidebar_label: "Twitter Firehose source connector" ---- - -:::note - -You can download all the Pulsar connectors on [download page](pathname:///download). - -::: - -The Twitter Firehose source connector receives tweets from Twitter Firehose and -writes the tweets to Pulsar topics. - -## Configuration - -The configuration of the Twitter Firehose source connector has the following properties. - -### Property - -| Name | Type|Required | Default | Description -|------|----------|----------|---------|-------------| -| `consumerKey` | String|true | " " (empty string) | The twitter OAuth consumer key.

For more information, see [Access tokens](https://developer.twitter.com/en/docs/basics/authentication/guides/access-tokens). | -| `consumerSecret` | String |true | " " (empty string) | The twitter OAuth consumer secret. | -| `token` | String|true | " " (empty string) | The twitter OAuth token. | -| `tokenSecret` | String|true | " " (empty string) | The twitter OAuth secret. | -| `guestimateTweetTime`|Boolean|false|false|Most firehose events have null createdAt time.

If `guestimateTweetTime` set to true, the connector estimates the createdTime of each firehose event to be current time. -| `clientName` | String |false | openconnector-twitter-source| The twitter firehose client name. | -| `clientHosts` |String| false | Constants.STREAM_HOST | The twitter firehose hosts to which client connects. | -| `clientBufferSize` | int|false | 50000 | The buffer size for buffering tweets fetched from twitter firehose. | - -> For more information about OAuth credentials, see [Twitter developers portal](https://developer.twitter.com/en.html). diff --git a/docs/io-twitter.md b/docs/io-twitter.md deleted file mode 100644 index 5fd3f167c9cf..000000000000 --- a/docs/io-twitter.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -id: io-twitter -title: Twitter Firehose Connector -sidebar_label: "Twitter Firehose Connector" ---- -