Re: [GMG-Devel] Plugin for page view counter

[Christopher Allan Webber]

Ah, thanks for that clarification.

... I don't know much about Redis. ;)

will kahn-greene writes:

> I think there's a misunderstanding here. I wasn't suggesting we use 
> Redis (or some equivalent) to keep tack of each +1 and then periodically 
> go through all those +1 and individually write them to the db--that's 
> dumb. Instead, I was suggesting we use Redis (or some equivalent) to 
> keep track of the current total and periodically persisting the total to 
> the database.
>
>
> On 01/14/2013 03:26 PM, Christopher Allan Webber wrote:
>> That would probably be possible... it would require allowing plugins to
>> set up other connections like that (I guess would probably mean
>> registering some things in mg_globals if we were going to do things the
>> "right" way?).  We could probably also use celerybeat to occasionally
>> dump it to disk... I'm not sure how much work that is.
>>
>> Related to the celery thing, I assume if we end up queueing up a number
>> of tasks in celery to increment, it's still the same number of writes to
>> do so, it's just that the writes are not happening at page load time...?
>> That does seem like it would avoid holding up a page from loading,
>> though if it's a "more writes at a time than the database can handle"
>> thing, I suppose that issue isn't going away in this solution.  But it
>> probably would at least not throw a bottleneck on page loading...
>>
>> That's as far as I can think of this at the moment :)
>>
>> will kahn-greene writes:
>>
>>> Or adding something that does fast atomic counting. I've seen Redis used
>>> for that in a few places. Then you dump the results of that to the db
>>> periodically so it's persisted. Not sure if there's anything in the GNU
>>> MediaGoblin infrastructure already that can do this.
>>>
>>> /will
>>>
>>>
>>> On 01/14/2013 12:06 PM, Christopher Allan Webber wrote:
>>>> Yes, you're right, celery is a better solution to processing these
>>>> things!
>>>>
>>>> Nathan Yergler writes:
>>>>
>>>>> I'd like to advocate for *not* doing a database write on a page view,
>>>>> particularly not one that's going to be subject to a high degree of
>>>>> lock contention on a higher traffic site. Write-on-GET should be a
>>>>> signal to think about things carefully, as it imposes a limit on how
>>>>> you can scale the software. If I recall correctly the stack still has
>>>>> Celery in it; tossing a job into the queue and accepting some latency
>>>>> with the stats isn't free, but it would reduce the amount of
>>>>> contention.
>>>>>
>>>>> Also keep in mind that either proxying the media file to trigger a
>>>>> stats increment, or embedding it in the HTML surrounding the media,
>>>>> mean you can't do any caching in front of the application: every
>>>>> request needs to hit the application server.
>>>>>
>>>>> Best,
>>>>>
>>>>> NRY
>>>>>
>>>>> On Mon, Jan 14, 2013 at 7:36 AM, Christopher Allan Webber
>>>>> <address@hidden> wrote:
>>>>>> Tiberiu C. Turbureanu writes:
>>>>>>
>>>>>>> Thanks for the pointer, Chris!
>>>>>>>
>>>>>>> I need a way to increment the view counter only when the media file is
>>>>>>> loaded. How can I achieve this?
>>>>>>
>>>>>> I'm not sure what you mean.  Do you mean:
>>>>>>    - only when the page is loaded
>>>>>>    - or something more complex: only if someone clicks "play" on a song 
>>>>>> or
>>>>>>      video?
>>>>>>
>>>>>> The solution I was talking about involves number one.  As for number
>>>>>> two, that would be a lot more complex, but I imagine it could be
>>>>>> achieved via one of two things:
>>>>>>    - Easier solution: javascript callback on play that notifies the
>>>>>>      browser of playing starting.
>>>>>>    - Much, much trickier solution: you might be able to set up a proxy
>>>>>>      view before serving the actual video that first increments the file,
>>>>>>      but then tells the web server to send the video via X-Sendfile.  
>>>>>> That
>>>>>>      sounds too complex to me though, and might be especially hard if a
>>>>>>      browser "seeks" the media to read just a few frames to get a preview
>>>>>>      image or do some preloading, which it might do in some
>>>>>>      configurations.
>>>>>>
>>>>>>> Also, Sebastian Spaeth told me that at the moment there is no way for a
>>>>>>> plugin to alter the template of the media page, but a hook mechanism is
>>>>>>> being discussed. That why until the implementation is ready, I think I
>>>>>>> will go for a new page under a route like this
>>>>>>> /u/<string:user>/m/<string:media>/viewcount.
>>>>>>
>>>>>> Spaetz is right, there's no present solution to this.  After I go
>>>>>> through my merge-a-thon this week though, I'll be switching onto some
>>>>>> plugin development, and will be looking into a solution to allow plugins
>>>>>> to extend pages more easily.
>>>>>>
>>>>>>    - Chris
>>>>>> _______________________________________________
>>>>>> devel mailing list
>>>>>> address@hidden
>>>>>> http://lists.mediagoblin.org/listinfo/devel
>>>>
>>>> _______________________________________________
>>>> devel mailing list
>>>> address@hidden
>>>> http://lists.mediagoblin.org/listinfo/devel
>>>>
>>> _______________________________________________
>>> devel mailing list
>>> address@hidden
>>> http://lists.mediagoblin.org/listinfo/devel