caching - Github API Conditional Requests with paging -

context: let's want retrieve whole list of starred repositories given user periodically (ones per day, hour or few minutes).

there @ least 2 approaches that:

1) execute https://api.github.com/users/evereq/starred , use url rel='next' in 'link' response headers next page url (we should till no "next" page in response, mean reach end). seems recommended approach (by github).

2) iterating 'page' parameter (from 1 infinite) using https://api.github.com/users/evereq/starred?page=xxx till 0 results in response. ones 0 results, finish (not recommended because example instead of page numbers github can move "hash" values. github did api operations.).

now, let's want make sure use conditional requests (see http://developer.github.com/v3/#conditional-requests) save our api usage limits (and traffic, trees in world, etc.).

so add example 'if-none-match' our requests headers , check if response status 304 (not modified). if so, means nothing changed our last request. works ok.

the issue have in 1) , 2) above, related way how detect when stop not working anymore when use conditional requests!

i.e. approach 1), don't link response headers @ when use conditional requests. need execute 1 more request page bigger page have etag , see return 0 results , know done. way "waste" 1 request github api (because miss conditional requests headers).

same approach 2), have 0 responses in every request status 304... again, know done, need make @ least 1 additional request return 0 results.

so question is: when conditional requests fact github api not send link response header (at least queries using etag result status 304) how know when stop paging? bug in github api implementation or miss something?

we don't know maximum page number, when stop should execute 1 more "waste" request , check if 0 results back!

i can't found how query github total count of starred repositories (so can calculate how many pages should iterate in advice), same responses not include "x-total-count" know when stop using simple math pages count.

any ideas how save 1 ('end') request , still use conditional requests?

if 1 request per day, it's ok accept such waste, if such request ones per minute? use api usage limits!

update

well, after few more tests, see following "rule" (can't found anywhere in docs, note sure if rule or assumption): if user star new, result every requested page contains different etag value compared previous , not have status 304 anymore! means it's enough request first page , check status. if 304 (not modified), not need check next pages, ie done nothing changed in page. correct approach or coincidence?

we indeed return pagination relations in link response header when content has changed 1. since don't support since parameter call, you'll need sort recent results , maintain client-side cursor last known id or timestamp (based on sort criteria) , stop paging when shows in paginated results. conditional requests let know if page 1 has changed.

we haven't settled on way return counts on our listing methods, low-tech solution set page size 1, grab rel=last link relation , check page parameter value.

hope helps.