chunking

I was under the impression that the lazy seqs were always chunked. => (take 1 (map #(do (print \.) %) (range))) (................................0) As expected 32 dots are printed because the lazy seq returned by range is chunked into 32 element chunks. However, when instead of range I try this with my own function get-rss-feeds , the lazy seq is no longer chunked: => (take 1 (map #(do (print \.) %) (get-rss-feeds r))) (."http://wholehealthsource.blogspot.com/feeds/posts/default") Only one dot is printed, so I guess the lazy-seq returned by get-rss-feeds is not chunked. Indeed: => (chunked-seq

I have been looking for an elegant and efficient way to chunk a string into substrings of a given length in Ruby. So far, the best I could come up with is this: def chunk(string, size) (0..(string.length-1)/size).map{|i|string[i*size,size]} end >> chunk("abcdef",3) => ["abc", "def"] >> chunk("abcde",3) => ["abc", "de"] >> chunk("abc",3) => ["abc"] >> chunk("ab",3) => ["ab"] >> chunk("",3) => [] You might want chunk("", n) to return [""] instead of [] . If so, just add this as the first line of the method: return [""] if string.empty? Would you recommend any better solution? Edit Thanks to

I need a way to upload large files (600 mb to 4 gb) in an asp.net mvc website. Currently I am using swfupload ; it works well enough, but it is a huge hit on the webserver because it sends it in one big upload, plus I have to set it in the web.config to allow that huge of a file, which is a huge security risk. In the past when I was doing web forms development I used Neatupload which breaks up the file into chunks and uploads them individually. I am looking for a way to upload large files in mvc that uploads via chunking it up. Any ideas on how I could do this? Silverlight File Upload Solmead

I wonder if anyone could have a look at the following code and minimal example and suggest improvements - in particular regarding efficiency of the code when working with really large data sets. The function takes a data.frame and splits it by a grouping variable (factor) and then calculates the distance matrix for all the rows in each group. I do not need to keep the distance matrices - only some statistics ie the mean, the histogram .., then they can be discarded. I don't know much about memory allocation and the like and am wondering what would be the best way to do this, since I will be

My exact scenario is inserting data to database in batches, so I want to accumulate DOM objects then every 1000, flush them. I implemented it by putting code in the accumulator to detect fullness then flush, but that seems wrong - the flush control should come from the caller. I could convert the stream to a List then use subList in an iterative fashion, but that too seems clunky. It there a neat way to take action every n elements then continue with the stream while only processing the stream once? Misha Elegance is in the eye of the beholder. If you don't mind using a stateful function in

I have a case where I need to transfer large amounts of serialized object graphs (via NetDataContractSerializer ) using WCF using wsHttp. I'm using message security and would like to continue to do so. Using this setup I would like to transfer serialized object graph which can sometimes approach around 300MB or so but when I try to do so I've started seeing a exception of type System.InsufficientMemoryException appear. After a little research it appears that by default in WCF that a result to a service call is contained within a single message by default which contains the serialized data and

I'm trying to chunk a sentence using ne_chunk and pos_tag in nltk. from nltk import tag from nltk.tag import pos_tag from nltk.tree import Tree from nltk.chunk import ne_chunk sentence = "Michael and John is reading a booklet in a library of Jakarta" tagged_sent = pos_tag(sentence.split()) print_chunk = [chunk for chunk in ne_chunk(tagged_sent) if isinstance(chunk, Tree)] print print_chunk and this is the result: [Tree('GPE', [('Michael', 'NNP')]), Tree('PERSON', [('John', 'NNP')]), Tree('GPE', [('Jakarta', 'NNP')])] my question, is it possible not to include pos_tag (like NNP above) and only

I am (still) attempting to upload large files <200mb via a html form using php. During my research into this I have come across the term "chunking", I understand that this process can break the file into handy sizes such as 5mb and reassemble them into the full file at the server side. My problem seems to be where I can begin? I seem unable to find the correct resources by googling (Or perhaps I'm suffering from not knowing which terms to search for). So what I'm hoping for today is a chance to educate myself with the basics, a direction in which to look would be very helpful. I don't really