Java紫微斗數排盤open source library

其實我一早就想把Fortel的core部份open source的了,不過現在才比較有空整理一下project,寫一點test case及doc。

Github: https://github.com/airicyu/Fortel

 

Fortel

Java紫微斗數排盤Library

Author: Eric Yu


Samples

排盤

排盤:一九五二年十二月十五日早子時天盤,男性

JSON Output(Formatted):


檢查宮垣

排盤:一九九零年三月十一日午時地盤,男性

檢查命盤命宮是否: 會見廉貞, 並且同時”天魁或天鉞同宮”或”不見化忌”

Output:


檢查命盤命宮是否: 會見廉貞, 並且同時”天魁或天鉞同宮”或”不見化忌”

Output:


JavaDoc:

You can view the Javadoc page at “\fortelcore\javadoc\index.html”

[Nodejs] FB Page post comment/reply event engine

岩岩寫左隻FB Page post comment/reply event engine既Nodejs module。

個module目的係幫FB page owner去mon住個FB page係咪有新comment,然後generate event & 俾個位d人自己去寫callback logic。

實際做法就係佢背底會行個schedule job去call FB graph API黎check post既new comments。

NodeJS module for keep track Facebook Page Posts New Comments

NodeJS module “fb-page-comment-event”

This module can keep track your Facebook page posts’ first level comment events. Hence, if someone commented on your post, you will be notified by an event.

This module support running an event-engine which would manage a background schedule job to check new comment and generating events. This module also support raw APIs which you can check new comments and digest as events by yourself via APIs.

 

Related Pages

Github page: https://github.com/airicyu/fb-page-comment-event
NPM: https://www.npmjs.com/package/fb-page-comment-event

 

Description

This module can keep track your Facebook page posts’ first level comment events. Hence, if someone commented on your post, you will be notified by an event.

This module support running an event-engine which would manage a background schedule job to check new comment and generating events. This module also support raw APIs which you can check new comments and digest as events by yourself via APIs.


Running the event engine

You may use the lib.pageCommentEventApp(options) API to get a event engine app and then start it with run function. This event engine app would manage a schedule job to auto pull data.

Sample:

  • Remarks: pageId is the facebook page ID. postId is the facebook page post ID of post which you want to monitor its comment. accessToken is the Facebook page access token(remember to use the long live token) which Facebook page owner can generate in developer dashboard.

Sample console log output:


(DIY) Use the APIs to check new comments and generating events

Sample:


event engine APIs

let app = lib.pageCommentEventApp({accessToken, pullInterval})

Description:

Initializing the event engine app.

parameters:

  • accessToken: The required accessToken(need permission manage_pages) for the API calls.
  • pullInterval: The sleep time(ms) interval between each scheduled page post new comment checking.

return:

The event engine app.

app.registerMonitorPost({pageId, postId})

Description:

Registering a page post which the event engine would tracking for new comments

parameters:

  • pageId: The page ID.
  • postId: The post ID.

app.run(eventsCallback)

Description:

Start running the event engine and register the new-comment-events callback. Once users write new comments, new-comment-events would be fired and handled by the callback.

parameters:

  • eventsCallback: The batch new-comment-events handling function.

app.stop()

Description:

Stop the event engine


DIY APIs

let queryPostCommentAgent = lib.api.getQueryPostCommentAgent(options)

Description:

Initialize a agent for query post comments.

parameters:

  • options: object with attribute accessToken which holding the required accessToken(need permission manage_pages) for the API calls.

return:

The agent object.

let postCommentFetcher = lib.api.getPostCommentFetcher(queryPostCommentAgent)

Description:

Initialize a fetcher for fetching new post comment.

parameters:

  • queryPostCommentAgent: The agent object which come from lib.api.getQueryPostCommentAgent(options)

return:

The fetcher object.

let postCommentObj = await postCommentFetcher.fetch(postObjId, since)

Description:

Use the fetcher object to fetch post object with comments. We can use this API to get all new comments under the target post which those comments are created since the since time.

parameters:

  • postObjId: The post object ID which is in format of ${pageId}_${postId} in Facebook Graph API.
  • since: The timestamp in second

return:

The post object with comments (All comments child objects are new comments).

let postDigestor = lib.api.getPostDigestor();

Description:

Getting a digestor object which can turn the post-comment object from postCommentFetcher.fetch() result into new-comment-events.

return:

The digestor object.

let newCommentEvents = await postDigestor.digest(postWithNewComments)

Description:

Use the digestor object to digest post-comment object into new-comment-events

parameters:

  • postWithNewComments: The post object with attaching new comments.

return:

New comment event objects

Airic API Gateway

Just for fun.
最近斷斷續續前後用左幾個星期左右,自己用NodeJS寫左個REST API Gateway。
 
簡單講隻REST API Gateway做d乜就是:
有個API config server,可以o係上面register隻app+import個swagger,
之後就可以create client同reg API key,
之後就可以用d API key經gateway去call果d REST API。
 
隻Gateway帶黎既benefits係:
– 隔左一層as protection layer,
– 有得落Quota rule去set quota (quota rule =/= spikearrest),
– 會收集到usage stat (1minute bucket),事後可以睇得到某時間內咩人call過邊個app邊個rest operation幾多次。
 
 

Project code name:

Airic API Gateway
 
 
 

基本Features:

 
– 冇UI…(其實依個唔係feature…)
因為我好怕攪frontend野太煩sosad, 所以直頭完全冇攪UI。
CRUD app API﹑CRUD client﹑register API key全部management operation靠REST API去做。
 
– app API definition係食swagger。
 
– support quota rule (1m, 5m, 1h, 1d, 1week, 1Month)。
quota rule可以apply落app level, operation tag level, 或者operation level。(app/tag/operation係跟據swagger spec而判斷)
 
– support對應同一app之下,唔同client分別用唔同quota plan (可以同時apply多個quota plan)
 
– 會log低API usage statistic & 可以用REST API query返出黎,睇到每隻app/operation/client o係某段時間內call左幾多次之類。
 
 

Performance

 
load test今日有試過下。
local PC用jmeter做load test (200 Testing thread), nodejs cluster x 4, 有enable 3條quota rule。
 
throughput都做到~2800 req/s。
random take sample既純粹入gateway processing直至response完成時間(而唔計proxy call背後API既時間)大約係0.x ms至10ms之間。
 
之後我再試多次,code果度直頭完全唔proxy call後面API,直接response返去,咁就真係可以計到純粹gateway processing直至response既時間& throughput。
咁樣load test係可以去到~5200 res/s。
random sample 入gateway到response時間都大約係0.x至10ms之間。
 

project status:

 
算係做左prototype,functionally基本叫做workable…
不過test我都只係人手test冇mocha unit test, 亦都冇寫doc。
(遲D執返好晒D野少少, 得閒先放上github吧~XD)
 
 

Component有邊幾樣野?

1) API config server – 儲住App﹑API config definition既核心。
2) API gateway – 就係隻gateway
3) API stat server – 儲住usage stat
(原本諗住寫多隻key server由config server拆出黎, 但我發覺key同config好難拆得開,我懶+怕煩就冇攪喇)
 
 
 

Database用左乜

1) App Config可以用MySQL或者Mongo或者Memory (好似Mongo方便dd, 不過我MySQL都係用json type儲, 所以差唔多)
 
2) API key可以用MySQL或者Mongo或者Memory (MySQL或者Mongo都冇所謂)
 
3) API call quota可以用Redis或者Mongo或者Memory (Prefer Redis, 好似快好多咁, 依part是用key-value lookup的)
 
4) API usage stat可以用MySQL或者Memory (Prefer MySQL, 只用來事後Aggregate, 其實用Mongo做都ok不過我冇整到)
 

收工交通消息Chrome Extension

icon

相關連結

Chrome Store page: https://chrome.google.com/webstore/detail/收工交通消息/amnocbjeifmlcogkicidpemmfjeimhdf

Github: https://github.com/airicyu/traffic-news-bot

介紹

這是一個關於收工時間交通消息notification的chrome extension。(只限於香港)

當你設定了收工時間之後,在星期一至五每日收工前的一小時內,這個工具就會把最新的交通消息更新推送給你。 你可以在設定頁面選擇是否推送,或只選擇推送有特定tag的交通消息。

另外,我們在顯示頁面也加入了Google map live traffic及運輸署全港主要道路交通情況的link讓你可以快速地看到現在的香港最新的路面交通情況。

 

使用方法

睇消息

裝左之後,右上plugin果度有依個extension既icon。 你可以click個icon打開popup page,就會睇到即時交通消息。

設定放工時間,等佢識早一粒鍾幫你去Check特別交通消息

popup page右上有”選項”,你click入去可以set你個放工時間(預設18:00)。 之後佢就會識計岩你放工前一小時內幫你check交通消息。
(預設下午6點收工)

Enable/Disable消息推送

如果你不喜歡消息推送,也可以在選項頁面disble這功能。
(預設enable消息推送)

關注特定交通消息的tag

如果你只想關注特定交通消息既tag (e.g. 吐露港公路 及 荃灣區),你可以去選項頁面的推送消息過濾設定。 首先取消”包括所有消息”,然後再click “吐露港公路” 及 “荃灣區”。
(預設是”包括所有消息”,會把所有消息都包括在內)

 

Screenshot

Popup頁面:

Popup頁面

選項頁面:

Push Notification頁面

Push Notification頁面:

Push Notification頁面

 

rest-in-contract – nodejs module for REST API Contract server

rest-in-contract

Project Page

Project Status

Currently, the project is in beta version (v0.x.x).

The basic Contract Server module is done to support basic usage of API Contract stubbing & testing. But some builtin feature is not done yet. (e.g: Suppoting more middleware functions in the contract script)

Since it is still beta version, we are not finalized the v1.0 in-the-box features yet.

Roadmaps

  • Add Unit tests
  • Update documents
  • Database Storage
  • Authentication
  • Support Plugins
  • Java/nodejs test integration client
  • Study on integration with Swagger

What is rest-in-contract

rest-in-contract is a product to let you embrace Consumer-driven contracts. It is REST in nature so that it fits for integrating with all kind of programming languages. For more detail about Project rest-in-contract, you may have a look in our Project rest-in-contract’s Homepage for detail introduction.

Slideshare: Basic Concepts & Flows

 


Samples

Hello world

Starting server:

Project rest-in-contract

Project rest-in-contract

Project Page

Related Projects

 

What is rest-in-contract

Consumer-driven contracts

rest-in-contract is a product to let you embrace Consumer-driven contracts. It is REST in nature so that it fits for integrating with all kind of programming languages.

Story for REST API providers/consumers in Consumer-driven contracts

For REST API providers:

REST API providers can write API contracts to describe their REST API request/response formats. They can then use the contracts to do contract testing against their API implementations.

For REST API consumers:

REST API consumers can use the API contracts to setup stubs for local testing or drafting of API contracts.

 

Slideshare: Basic Concepts & Flows

 

How does rest-in-contract different from other Consumer-driven contracts solution?

REST in nature, cross language, easy integration

There are many Consumer-driven contracts solution existing but many of them are SDK libraries or embedded solution for stubbing or testing which may fixed in a certain language. rest-in-contract is designed in a prespective that we do not want a language fix-in solution.

rest-in-contract is a node modeule to setup a lightweight agent server which API providers/consumers can kick to start in their environment easily. No matter doing contract testing(For provider) or API stubbing(For consumer), you can always do them by calling rest-constract agent server’s REST API. That’s why rest-in-contract is a cross language solution for Consumer-driven contracts.

Thanks for REST in nature, it is very easy to do integration with DevOps. No need maven or gradle build. You just need node.js(v7+) installed in your environment to kick the server. All later interactions are REST API call which you can call with curl or any other HTTP client tools.

Contract as file

Some Consumer-driven contracts solutions may let you wiring stub servers by SDK methods. Hence, the contract is writen as embedded code. Such way has difficulties for supporting different kind of programming languages.

Instead, we think that contract should be defined in a less coupling way that can be separated from your business logic codes.

The contract in rest-in-contract is described in JS script format which exporting an Contract object. We supporting some middleware function call in the contract. It also support regular expression, jsonpath etc.

An example of contract file would be like this:


 

Although the contract file is written in javascript in syntax, but you can just treat them as general files and stored in your projects. It is because that your code/application would never necessary to directly interact with the contracts. You can always pass them to the Contract Server to let it do its job.

 

What are the possible architecture configurations of rest-in-contract?

Architecture Components

Contract Server is a server instance which supporting Contract testing and stubbing by REST API. It is typically storing and reading contracts in local storage.

Contract Agent Server is actually a Contract Server. The different is that it read contracts from remote contract repository instead of local storage.

Architecture Configurations

We imagined that that there may be two architecture configurations of using rest-in-contract.

  1. Centralized Contract Server architecture

In this mode, there would be a centralized Contract Server which serving all API provider and consumers. It is the centralized storage server for persisting all contracts in database.

API providers & consumers would setup their own Contract Agent Server in their own environment which get the contracts remotely from centralized Contract Server through REST API. Then they would use their local Contract Agent Server to do contract testing or stubbing.

  1. Decentralized Contract Server architecture

In this mode, API providers would keep contracts in their own way. For examples, if their application is on Github, they may put the contracts under a folder in their source codes. API providers can kick Contract Server in local environment to do contract testing.

On the other side, API consumers can checkout the project from Github in order to get the API contracts. Then API consumers can kick Contract Server in local environment to do stubbing.

 


 

Story walkthrough

Beginning of the Story: John(API Provider) wrote an application Foo which provide REST API.

John wrote an application “Foo” which the base application URL is “http://example.com/foo“. It has a version path “/v1.0”. And the API endpoint url is “/hello”.

The API has such format:

Request:

Response:

The request body has an attribute name with a string value.

 

Next Story: Mary(API Consumer) is writing an application bar which want to consume API from John’s API.

Mary want John to enhance his API to include a new integer attribute “age” in request and output it in response like this:

Request

Response:

Hence, Mary and John have a discussion and drafted an API contract like this:


 

This single contract would be used by both John and Mary. For John, he would use this contract to do contract testing against its implementation. For Mary, she would use this contract to generate stub for local testing.

To support both use case of contract testing and stubbing, they need to define value(stub(...), test(...)) in the contract. The meaning of value(stub(...), test(...)) in contract is that, the certain values which would be used for generating stub and contract testing are different.

Why different values for stubbing and testing?

For Mary, she wants stub. The stub would accept any “name” attribute which is in [a-zA-Z ]* pattern. It means that Mary can send a request to the stub with “name” set as “Susan” or “Sam” or any other valid names…… And the response should correctly showing the same name in the request. The “name” attribute should support a flexible value so that Mary can test more dynamically instead of a always hard coded value. Hence, it is represented by a regular expression pattern regex("[a-zA-Z ]*").

For John, he wants API test. The API test just need to define a test value which is used for testing. (Actually he can use regular expression pattern as well, but here is just for demo) Hence, a hardcoded test value “John” is used. And the response value is also hardcoded test value.

 

Next Story: John use the contract to generate testing endpoint to test against the API contract

Firstly, John start a local Contract Server with port 8000.

And then he create(register) an App in Contract Server by this REST API call:

Assume the app ID is “80a69a44-3f3b-48c1-a7d1-b34b89117e75”.

For this app, John create(register) an App version “2.0” in Contract Server by this REST API call:

And then John create(register) the API contract in Contract Server by this REST API call:

(Assume the contract ID is “94923fbd-9092-4a46-ad65-0d8a2e2f551e”)

And then John can update “foo” ‘s API version 1 to include this API contract. He can do it by calling this REST API:

And then John can use Contract Server’s REST API to trigger contract testing against app “Foo” in his local environment (Port 8001). Once triggered, Contract Server would build mock requests according to the API contract and then send to the target API endpoint. The testing result and the whole Request/Response context information would be returned.

The Contract Test REST API would be like this:

Request:

The Response may look like this:

Next Story: Mary use the contract to kick start stub server

Thie time, Mary start a local Contract Server with port 8000. And then she create the App(John’s app “Foo”), Version and Contracts just like the last story. Again, it is done by REST API calls.

But at the last step, thie time she would not call the API contract testing operation ( /api/v1/apps/{{appId}}/wiretests ). Instead, she want to wire a stub server. Hence, she would call the API wire stub operation ( /api/v1/apps/{{appId}}/wirestubs ).

After that, a stub server which responses according to the API contracts is started at local port 8001.

And then, Mary can test the hello API with the stub server. She can send a request like this:

And she would get this response:

After Mary’s testings, Mary can shutdown the stub server by this API:

Or, she can shutdown the whole local Contract Server instead.

 


Credit to Spring-Cloud-Contract & WireMock

We have to give credit to Spring-Cloud-Contract & Wiremock because this project is inspired by them. We like Spring-Cloud-Contract’s design and usage of value(stub(...), test(...)) so we bring it in rest-in-contract. We also appreciate Wiremock which showing us an well-made product for case study/feature analysis to help us make our own new product.

[Article sharing] A lesson for developers to revise on CSRF attack protection

Here is the article: Defense Against the Dark Arts: CSRF Attacks
This is a good article which telling a humor story about how to protect web application from CSRF (Cross-site request forgery) attack.
I think it is 120% worth for web app developers to read for 溫固知新, no matter you are frontend or backend developer.

I suggest to read the whole story, but for those TL;DR , I can share the summary here:

 

  1. Introduced an authentication system to prevent attackers from impersonating users.
  2. Used cookies to do this in a way that does not require two HTTP roundtrips (with a loading spinner in between) to view pages with private information, like a page listing a user’s private messages.
  3. Defended against <img src="some-endpoint-here"> GET CSRF attacks by requiring that endpoints which make changes to things use HTTP verbs other than GET. (In this case, we used POST.)
  4. Defended against <form> POST CSRF attacks by checking that the Origin and/or Referer headers match hogwarts.edu (and rejecting the request if neither header is present).
  5. Added a second line of defense against future potential Origin and/or Referer vulnerabilities by requiring that the Content-Type header be set to application/json.

 

Dear developers, have a nice day~!

stateful-result – nodejs module for represent operation result with status code

Nodejs stateful-result module

This module can be used for representing an operation(e.g: function return) result with a meaningful status code. By default, we suggest to make use of HTTP status code. Hence, code 200 for status of operation OK, and code 404 for status of operation target not found, etc.

Links

Github: https://github.com/airicyu/stateful-result
NPM: https://www.npmjs.com/package/stateful-result

Samples

Sample 1: Getting success result (code 200 OK)

console output

Description:

Simple enough, isn’t it?

Sample 2: Getting fail result (code 404 Not found)

console output

Description: The error object is an extended error object which has the “code” attribute. In this case, the code is 404.

Sample 3: Get result if success result or throw error if returning fail result (code 404 Not found)

console output

Description: This time we called method “getOrThrow”. If the result is success, the result would be returned as previous samples. If the result is fail, the error object would be thrown.

一些寫library的practices分享

我寫library/module通常有一d practices。

1) 我多數是將data store的底層abstract。
並且default使用memory data store。(e.g:用個hashmap儲住就算)

我咁做唔係因為懶/求奇。
而係因為我自己作為developer理念係minimum-viable就是最簡單,就是最好。
面向developer,其實最好就係用最快最簡單最少step方法就可以俾developer taste得到。
default使用memory data store,就至少唔駛setup database。

但我通常亦都會留位俾developer去switch用其他data store implementation。
我通常都會另外做一個堅database data store implementation 俾佢next step即刻可以switch到。
最近寫的code,我有時會再另外再做一個remote REST resource data store implementation。

2) 我多數都會做埋self contain的minimum-runnable helloworld samples。
我擺上github果堆library,我都一定會寫埋readme Document。
但developer好多時同時好需要sample as a reference去抄抄貼貼黎用,會快過佢地自己齋對住readme doc/API doc黎由零砌起。
砌一d minimum-runnable helloworld samples,佢可以踢著黎試/trace/改code黎自己去研究點玩點用。

3) Document, test case, 依d基本野唔駛講。